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ABSTRACT 

This  study  examines  the  accuracy  of  an  Army  helicopter 
pilot  workload  measuring  model  called  the  Task  Loading  Model. 
The  model  is  a  submodel  of  the  Army-NASA  Aircrew/Aircraft 
Integration  Program's  Man-Machine  Integration  Design  and 
Analysis  System.  The  model's  workload  level  output  was 
correlated  with  the  subjective  workload  measurements  of 
several  groups  of  pilots  evaluating  a  variety  of  flight  tasks. 
Seventy-one  Army  aviators  completed  surveys  requiring  scaled 
ratings  and  paired  comparisons  of  workload  related  to  common 
flight  tasks  conducted  during  typical  missions.  Their 
responses  were  examined  for  internal  consistency  and  pooling 
by  means  of  nonparametric  tests.  Aviator-supplied  data  was 
found  to  be  robust  and  reliable.  Pooled  response  data  was 
correlated  with  model-generated  data  to  determine  the  accuracy 
of  the  model.  Results  of  this  study  show  that  the  Task 
Loading  Model  is  presently  inadequate,  but  displays  promising 
trends  and  should  be  further  refined. 
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INTRODUCTION 


A.   BACKGROUND 

1.   Operator  Error  and  the  Design  Process 

Operator  error  is  the  primary  cause  of  many  aircraft 
mishaps.  "Even  if  some  divine  power  could  guarantee  that  all 
mechanical  and  electrical  systems  used  in  aviation  were  100% 
reliable,  there  would  still  be  accidents  due  to  human  error" 
(Kantowitz,  1988,  p. 158).  This  is  especially  true  in  the 
demanding  environments  associated  with  military  aviation.  As 
technological  advances  raise  the  performance  levels  of  modern 
military  aircraft,  the  mission  employment  expectations  placed 
upon  the  aircraft  and  their  operators  also  rise.  As  a  result, 
pilots  encounter  more  difficult  operational  environments, 
experience  higher  workload  demands,  and  freguently  surpass 
their  human  performance  capabilities. 

There  is  a  concerted  effort  among  design  engineers  to 
reduce  operator  workload  through  cockpit  automation.  However, 
a  large  gap  exists  between  the  desired  effects  of  automation 
and  its  actual  ramifications.  Instead  of  lowered  operator 
workload,  many  well  intentioned  designs  result  in  prolonged 
training,  illogical  tasking,  and  excessive  demands  on  pilot 
adaptability.  (Army-NASA,  1990,  p. 2) 
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2.   Army-NASA  Design  Tool:  MIDAS 

The  U.S.  Army  and  the  National  Aeronautics  and  Space 
Administration  (NASA)  are  working  together  to  create  a  human 
performance  model  that  overcomes  the  problem  of  good 
intentions  and  poor  results.  The  Army-NASA  Aircrew/Aircraft 
Integration  Program  (A3I)  is  developing  a  computer  simulation 
of  aircraft  crewstations  and  their  operators.  The  simulation 
is  to  serve  as  a  tool  that  can  help  ensure  that  human  factors 
considerations  are  included  in  system  design. 

The  Man-Machine  Integration  Design  and  Analysis  System 
(MIDAS)  is  one  part  of  the  A3I  program's  efforts.  The  MIDAS 
workstation  presently  simulates  an  AH-64  Apache  helicopter, 
its  pilot,  and  the  environment  in  which  they  operate.  MIDAS 
includes  a  variety  of  simulation  submodels  in  areas  such  as 
anthropometry,  vision,  cockpit  display  layout,  the  pilot's 
cognition,  and  task  loading.  The  synchronization  of  the  MIDAS 
submodels  results  in  a  very  capable  design  tool.  (Army-NASA, 
1990) 

MIDAS  developers  envision  an  interactive  computational 

system  such  as  that  illustrated  in  Figure  1.   Use  of  such  a 

system  will  improve  the  current  design  process  in  several 

ways: 

1.  It  enhances  the  engineer's  efficiency  and  creativity.  He 
or  she  can  configure  a  cockpit,  test  it  with  a  human 
model,  and  immediately  evaluate  the  design's  suitability 
in  a  variety  of  environments. 


2.  Human  factors  considerations,  often  ignored  during  the 
initial  phases  of  design,  are  a  significant  aspect  of  the 
feedback  produced  by  MIDAS. 

3.  Extensive  use  of  computer  modeling  will  reduce  the 
expensive  development  of  faulty  prototypes,  thus  reducing 
the  research  and  development  costs  associated  with  a 
procurement  program. 

4.  Although  now  configured  as  an  Apache  helicopter,  MIDAS 
will  be  applicable  to  any  man-machine  vehicular  system. 

(Army-NASA,  1990,  pp. 1-6) 


Figure  1.  MIDAS  Workstation  Concept  (Army-NASA,  1990,  p. 8) 
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3.   MIDAS  Task  Loading  Model 

The  measurement  of  pilot  workload  is  an  important 
aspect  of  the  aircraft  design  process.  "When  workload  is 
excessive,  errors  arise  from  the  inability  of  the  human  to 
cope  with  high  information  rates  imposed  by  the  environment. 
When  workload  is  too  low,  the  human  is  bored  and  may  not 
attend  properly  to  the  task  at  hand,  also  leading  to  error" 
(Kantowitz,  1988,  p. 159). 

The  Task  Loading  Model  (TLM)  is  one  high  resolution 
facet  of  MIDAS.  The  TLM  manipulates  state  variables  related 
to  the  operator  and  the  aircraft  to  calculate  the  operator's 
relative  workload.  The  TLM  output  classifies  the  pilot's 
individual  flight  tasks  and  task  demands  across  the  visual, 
auditory,  cognitive,  and  motor  dimensions.  The  classifi- 
cations and  calculations  generate  workload  data  that  designers 
may  use  to  determine  the  level  of  human  performance  required 
for  a  specific  system  configuration.  (Staveland,  1990) 

Four  dimensions  of  human  performance  are  measured  by 
the  TLM.  These  four  dimensions  represent  the  visual, 
auditory,  cognitive  and  motor  activities  of  a  pilot.  The  TLM 
subdivides  the  four  dimensions  into  several  elements, 
including  near  and  far  for  a  visual  activity,  salient  and 
masked  for  an  auditory  activity,  planned  and  unplanned  for  a 
cognitive  activity,  and  gross  and  fine  for  a  motor  activity. 
Each  task  is  classified  according  to  the  four  dimensions  and 

4 


the  elements  within  the  dimensions.   The  TLM  then  generates 
workload  values  along  each  dimension.  (Staveland,  1990) 

B.   TLM  PROBLEMS 

Several  problems  and  uncertainties  currently  are 
associated  with  the  TLM.  First,  the  nature  of  human  workload 
is  not  fully  understood.  Although  the  model  incorporates 
"current  research  in  multiple  resource  theory,  scaling, 
workload,  and  perception"  (Army-NASA,  1990,  p. 14),  limitations 
in  the  field  of  behavioral  psychology  still  limit  the  model. 
Second,  current  workload  measurement  mechanisms  cannot  capture 
all  aspects  of  task  loading  and  human  performance.  Thus, 
behavioral  scientists  skeptically  view  attempts  to  compile 
workload  data.  Third,  the  TLM  output  reguires  verification 
with  credible  workload  data  to  insure  that  it  reflects 
reality.  However,  even  "suspect"  helicopter  pilot  workload 
data  suitable  for  comparison  with  the  TLM  output  is  not 
available. 

There  is  a  critical  need  to  verify  the  TLM  output  of 
workload  values  by  comparing  them  with  values  obtained  by 
other  means.  This  is  true  despite  the  present  lack  of 
standard  workload  measurement  technigues.  "The  workload 
problems  of  today  will  not  wait  until  scientists  develop 
perfect  models.    Fortunately,  even  incomplete  models  are 


useful  in  developing  practical  specifications"  (Kantowitz, 

1988,  p. 182)  . 

C.   PURPOSE  AND  SCOPE 

The  purpose  of  this  study  is  to  provide  data  that  can  be 
correlated  with  output  from  the  A3I  MIDAS  TLM.  The  study 
consisted  of  three  primary  procedures.  First,  subjective 
workload  data  was  collected  from  subject  matter  experts 
(helicopter  pilots)  by  means  of  a  survey  form.  Next,  the 
survey  data  was  analyzed  to  develop  a  robust,  valid  set  of 
workload  values.  Finally,  TLM  output  of  workload  values  was 
compared  with  survey  workload  data  to  determine  how  closely 
the  two  data  sets  correlated. 

Chapter  II  of  this  thesis  discusses  the  methodology  used  to 
collect  subjective  workload  estimates  from  helicopter  pilots. 
Chapters  III  and  IV  provide  the  data  analysis  techniques  and 
results.   Chapter  V  outlines  conclusions  and  recommendations. 


II.  METHODOLOGY 

A.  STUDY  OVERVIEW 

Verifying  the  reliability  of  the  TLM  output  was  a  six-step 
process.  The  data  used  for  the  study  came  from  a  survey 
administered  to  US  Army  helicopter  pilots  in  a  classroom 
environment.  This  method  of  data  collection  allowed  for  input 
from  71  subject-matter  experts  in  a  short  time  frame  and  at 
minimal  expense.   The  six  steps  of  the  process  were: 

1.  Identification  of  helicopter  flight  tasks  for  which  to 
determine  workload  values. 

2.  Construction  of  survey  forms  which  were  used  to  collect 
pilot  opinions  about  the  workload  levels  related  to  these 
tasks. 

3.  Determination  of  data  analysis  procedures. 

4.  Administration  of  the  surveys  to  Army  helicopter  pilots. 

5.  Analysis  of  the  workload  data  collected  via  the  survey. 

6.  Comparison  of  the  survey  data  to  the  TLM  output  of 
workload  values  related  to  the  same  flight  tasks. 

The  remainder  of  this  chapter  further  explains  the  first  four 

steps  of  the  above  procedure. 

B.  SCENARIOS  AND  TASKS 

1.   Flight  Scenario  Development 

Flight  tasks  for  which  workload  values  were  collected 
were   identified  through  an  analysis  of   flight  missions 
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commonly  carried  out  by  the  pilots  who  were  surveyed. 
Similar,  but  not  identical,  flight  scenarios  were  developed 
both  for  attack  and  for  scout  helicopter  pilots,  since  their 
missions  are  closely  related  but  vary  in  significant  ways. 
The  scenario  settings  used  for  the  survey  are  provided  in  the 
survey  forms,  examples  of  which  are  included  as  Appendix  A 
(scout  helicopter)  and  Appendix  B  (attack  helicopter) . 

Each  of  the  two  scenario  descriptions  was  developed  by 
a  pilot  with  experience  in  both  the  mission  and  the  type  of 
aircraft  "flown"  in  the  scenario.  Each  scenario  represents 
approximately  a  two-minute  slice  of  time  from  a  typical  attack 
or  reconnaissance  mission.  Tasks  occurring  during  the 
scenario  are  based  on  what  a  realistic  cockpit  crew  is 
expected  to  do,  under  the  assumption  that  the  survey 
respondent  is  manipulating  the  flight  controls.  This  creates 
a  single-pilot  situation  from  which  consistent  task  workload 
measurements  could  be  drawn. 
2.   Task  Selection 

The  specific  tasks  associated  with  each  scenario  were 
next  identified.  Of  the  more  than  20  maneuvers  conducted  by 
the  pilot  throughout  the  scenario,  nine  items  were  selected  as 
"individual  tasks,"  which  are  defined  as  discrete  activities 
carried  out  by  a  pilot  while  flying  (see  Tables  1  and  2) .  An 
activity  is  a  physical  or  mental  process  associated  with  a 
helicopter  flight  mission.  Activities  may  be  described  by  the 
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four  dimensions  of  human  performance  associated  with  the  TLM: 
the  visual,  auditory,  cognitive  and  motor  dimensions. 

Additionally,  the  nine  individual  tasks  were  merged 
into  six  "combined  tasks,"  also  listed  in  Tables  1  and  2. 
Combined  tasks  consist  of  discrete  events  that  overlap  along 
a  mission  time  line.  For  example,  Tell  crewmember  unmasking 
is  one  individual  task  and  Unmask  aircraft  is  another  individ- 
ual task.  However,  Tell  crewmember  "Unmasking"  while  unmask- 
ing  the   aircraft    is  a  combined  task. 

Since  workload  levels  collected  via  the  surveys  were 
to  be  compared  with  those  generated  by  the  MIDAS  TLM,  it  was 
important  that  the  tasks  included  in  the  survey  forms  consist 
of  a  potentially  wide  variety  of  workload  levels.  The  nine 
individual  tasks  selected  from  the  two  scenarios  met  this 
requirement.  As  a  whole,  they  encompassed  the  range  of 
loading  evaluated  by  the  TLM.  They  were  selected  intentional- 
ly to  verify  all  dimensions  of  the  model,  as  described  in 
Chapter  I . 

The  nine  individual  tasks  selected  for  the  attack  and 
scout  helicopter  pilots  were  exactly  the  same  or  parallel. 
For  example,  Task  4  for  both  groups  of  pilots  was  Hover. 
However,  Task  8  for  the  attack  helicopter  pilots  was  Switch 
master  arm  to  "arm,"  while  for  the  scout  pilots  it  was  Turn  up 
volume  on  ICS  control  panel.  Both  tasks  involve  reaching  with 
the  left  hand  and  adjusting  a  switch  or  knob  with  the  fingers. 


TABLE  1.   ATTACK  HELICOPTER  FLIGHT  TASKS 


TASK 
No. 

TASK 
CODE 

INDIVIDUAL  TASKS 

1 

Folw 

Follow  scout 

2 

Decl 

NOE  deceleration 

3 

Cktq 

Check  torque  meter  and  announce  reading 

4 

Hovr 

Hover 

5 

Tell 

Tell  crewmember  "Unmasking" 

6 

Umsk 

Unmask  aircraft 

7 

Lstn 

Listen  to  radio  transmission  from  scout 
(direction  to  fire,  range,  target) 

8 

Swch 

Switch  master  arm  to  "arm" 

9 

Intp 

Interpret  gunner's  position  of  TSU  based  on 
PSI  information 

COMBINED  TASKS 

1 

Cdecl 

NOE  deceleration  behind  scout 

2 

Ccktq 

Check  torque  meter  and  announce  reading 
while  NOE  decelerating 

3 

Cckhv 

Check  torque  meter  and  announce  reading 
while  hovering 

4 

Cumsk 

Tell  crewmember  "Unmasking"  while  unmasking 
aircraft 

5 

Clstn 

Listen  to  radio  transmission  from  scout 
(direction  to  fire,  range,  target)  while 
switching  master  arm  to  "arm" 

6 

Cintp 

Interpret  gunner's  position  of  TSU,  based 
on  PSI  information,  while  receiving  target 
handoff  from  scout 
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TABLE  2.   SCOUT  HELICOPTER  FLIGHT  TASKS 


TASK 
No. 

TASK 
CODE 

INDIVIDUAL  TASKS 

1 

Folw 

Follow  another  scout 

2 

Decl 

NOE  deceleration 

3 

Cktq 

Check  torque  meter  and  announce  reading 

4 

Hovr 

Hover 

5 

Tell 

Tell  crewchief  "Unmasking" 

6 

Umsk 

Unmask  aircraft 

7 

Lstn 

Listen  to  crewchief  describe  what  he  sees 

8 

Turn 

Turn  up  volume  on  ICS  control  panel 

9 

Detm 

Determine  present  location  (6  digit  grid) 
as  crewchief  holds  map 

COMBINED  TASKS 

1 

Cdecl 

NOE  deceleration  behind  another  scout 

2 

Ccktq 

Check  torque  meter  and  announce  reading 
while  NOE  decelerating 

3 

Cckhv 

Check  torque  meter  and  announce  reading 
while  hovering 

4 

Cumsk 

Tell  crewchief  "Unmasking"  while  unmasking 
aircraft 

5 

Clstn 

Listen  to  crewchief  describe  what  he  sees 
while  you  turn  up  volume  on  ICS  control 
panel  and  hover 

6 

Cdetm 

Determine  present  location  (6  digit  grid) 
as  crewchief  holds  map  and  you  listen  to 
him  describe  what  he  sees 
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C.   SUBJECTIVE  WORKLOAD  MEASUREMENT  TECHNIQUES 

1.  Selection  of  Measurement  Techniques 

Several  means  of  eliciting  workload  judgements  from 
pilots  were  considered.  The  desired  characteristics  of  a 
workload  measuring  mechanism  were  as  follows: 

1.  Reliable  and  credible  results  are  obtained. 

2.  An  appropriate  level  of  effort  is  required  of  the 
respondent. 

3.  The  resulting  subjective  measurements  can  be  quantified. 

4.  Administration  to  groups  of  15  or  more  is  easy. 

5.  Data  analysis  is  straightforward. 

Two  established  techniques  were  chosen  for  this  study, 
since  both  met  the  above  requirements.  They  were  subjective 
workload  ratinq  scales  and  the  psychometric  technique  known  as 
Thurstone's  Method  of  Paired  Comparisons  (ARI  Report  851, 
1989,  pp. 81-122).  Examples  of  both  of  these  techniques  are 
included  in  the  example  survey  forms  located  in  Appendix  A  and 
Appendix  B. 

2.  Rating  Scales 

Workload  rating  scales  require  the  respondents  to 
assign  a  numerical  value  (from  0  to  10  for  this  study)  to  each 
task,  thereby  indicating  its  level  of  difficulty.  The  use  of 
rating  scales  was  advocated  by  the  designer  of  the  MIDAS  TLM, 
resulting  from  his  experience  with  the  NASA-Task  Load  Index 
(TLX)  (Hart  &  Staveland,  1987).   The  TLX  is  a  highly  refined 
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subjective  workload  assessment  scaling  technique  based  on 
rating  scale  methodologies.  Rating  scales  meet  the  five 
requirements  listed  above  for  a  workload  measuring  mechanism. 
They  allow  for  any  number  of  subjects,  tolerate  missing 
responses,  and  facilitate  efficient  statistical  analysis.  A 
potential  drawback  is  that  raters  become  frustrated  when 
required  to  make  large  numbers  of  judgements.  Thus,  reducing 
possible  frustration  was  a  major  reason  for  including  only 
nine  tasks  on  the  survey  form  (Zatkin,  1983) . 
3.   Thur stone's  Method 

Thurstone ' s  method  utilizes  several  judges  who  compare 
each  item  in  a  set  against  each  of  the  other  items,  one  at  a 
time.  This  method  results  in  an  ordering  of  the  items 
according  to  a  relative  value  of  magnitude  interdependent  on 
every  other  compared  item.  For  the  workload  measurements, 
each  pilot  determined  which  of  two  compared  tasks  was  more 
difficult.  Judgements  were  made  for  each  set  of  two  until  all 
individual  or  combined  tasks  had  been  compared  with  all  the 
other  individual  or  combined  tasks.  From  these  comparisons  a 
scale  of  relative  task  difficulty  emerged.  (Thurstone,  1963) 

A  drawback  to  Thurstone ' s  Method  is  the  potentially 
cumbersome  number  of  pairwise  comparisons  required  of  the 
respondent.  To  compare  n  items  against  each  other  requires 
n(n-l)/2  judgements.  Workload  measurements  for  this  study 
required  36  comparisons  of  the  nine  individual  tasks  and  15 
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comparisons  of  the  six  combined  tasks.    This  number  of 
pairwise  comparisons  and  the  accompanying  scale  ratings 
provided  a  challenging,  but  not  overwhelming  endeavor  for  the 
pilots  participating  in  the  survey. 
D.   SURVEY  CONSTRUCTION 
1.   Survey  format 

Two  separate  survey  forms  were  used  to  measure 
helicopter  pilot  experience  with  workload  during  flight,  one 
for  scout  helicopter  pilots  and  the  other  for  attack  helicop- 
ter pilots  (see  Appendices  A  and  B) .  Each  was  designed  to  be 
completed  in  approximately  15-2  0  minutes.  The  12 -page  survey 
forms  included  an  introduction,  instructions  on  how  the  survey 
should  be  filled  out,  and  the  scenario  setting.  The  survey 
itself  was  divided  into  four  sections: 

1.  Individual  task  subjective  rating  scales. 

2.  Combined  task  subjective  rating  scales. 

3.  Individual  task  pairwise  comparisons. 

4.  Combined  task  pairwise  comparisons. 

Pilot  background  information  also  was  collected. 

Ratings  and  comparisons  were  randomized  within  each 
section  for  each  individual  survey  form.  The  four  sections 
also  were  shuffled  into  their  24  different  permutations,  to 
create  24  unique  formats.  Rosenthal  points  out  that  "very 
large  questionnaires  can  tend  to  discourage  the  subjects  from 
answering.  A  way  around  this  problem  is  to  vary  the  format  in 
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order  to  keep  the  subjects  interested"  (Rosenthal,  1984, 
p. 134).  Besides  maintaining  subject  interest,  randomization 
also  reduced  the  chances  that  judges  might  respond  according 
to  a  set  contextual  flow  of  the  survey  material  (Rosenthal, 
1984)  . 

2 .   Survey  Content 

The  first  page  of  each  survey  form  provided  background 
information  to  the  pilots.   Several  points  were  highlighted: 

1.  The  survey's  purpose:  attain  workload  information  to  aid 
in  the  improvement  of  future  Army  helicopter  cockpits. 

2.  Emphasis  on  candid  responses,  with  no  right  or  wrong 
answers. 

3.  Individual  anonymity  and  the  pooling  of  response  data. 

4.  The  uniqueness  of  each  survey  form  due  to  question 
randomizing. 

These  points  were  discussed  to  help  control  what  Rosenthal 
calls  "demand  characteristic:  the  subject's  perception  of  his 
or  her  role  and  of  the  experimenter's  hypothesis."  Clarifying 
the  pilots'  role  made  them  more  sensitive  to  the  aims  of  the 
survey,  thereby  motivating  a  response,  while  instilling 
confidence  in  their  ability  to  respond  appropriately. 
(Rosenthal,  1984,  p. 105) 

The  second  page  of  the  form  consisted  of  detailed 
instructions  indicating  how  to  respond.  Specific,  relevant 
examples  were  included.  This  was  followed  by  a  page  contain- 
ing a  one-paragraph  scenario  description  providing  the  context 
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within  which  the  pilot  was  to  respond.  The  setting  was  that 
of  a  typical  tactical  flight  mission.  Applicable  parameters 
such  as  crew  mix,  weather  conditions,  and  the  enemy  situation 
were  described.  The  forms  for  the  actual  ratings  and  compari- 
sons followed.  The  final  page  contained  a  short  questionnaire 
regarding  the  aviation  experiences  and  qualifications  of  the 
responding  pilot. 

The  two  survey  forms  were  reviewed  for  accuracy  of 
content  and  wording  by  two  Army  instructor  pilots.  They 
provided  helpful  feedback  by  recommending  current  aviation 
phraseology,  such  as  changing  "copilot"  to  "crewmember . "  They 
verified  the  sound  structure  of  the  scenarios  and  reported 
that  the  survey  was  appropriate  for  the  objectives  that  were 
to  be  met. 

3.   Minimizing  the  Social-Desirability  Response 

A  concept  from  the  field  of  behavioral  science  that 
influenced  survey  construction  was  what  is  referred  to  as  the 
"social-desirability  response  set"  (Rosenthal,  1984,  pp. 135- 
139) .  With  respect  to  helicopter  pilots,  this  mindset  would 
tend  to  cause  them  to  tailor  their  responses  so  they  would 
appear  to  be  competent  pilots  who  rarely  experience  demanding 
workload.  Three  techniques  were  utilized  to  minimize  the  "I 
can  handle  anything"  mentality  common  among  pilots.  First, 
anonymity  was  emphasized  so  that  no  individual  could  be  linked 
to  a  specific  response.  Second,  the  forced  choices  associated 
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with  Thurstone's  paired  comparisons  required  the  respondent  to 
specify  that  one  task  was  harder  than  another.  This  prevented 
respondents  from  claiming  that  none  were  difficult.  The  third 
technique  involved  the  interaction  of  the  survey  administrator 
with  the  pilots  who  were  completing  the  survey,  as  discussed 
in  the  next  section. 
E.  SURVEY  ADMINISTRATION 
1.   Survey  Respondents 

Surveys  were  administered  to  pilots  assigned  to  the 
Seventh  Infantry  Division  (Light)  located  at  Fort  Ord, 
California.  Pilots  from  the  2/9  Cavalry  Squadron  completed 
the  survey  on  29  March  1991.  Pilots  from  the  1/123  Attack 
Helicopter  Battalion  completed  the  survey  on  23  May  1991. 
Table  3  shows  the  number  of  respondents  by  category. 

A  third  group  of  respondents  consisted  of  Enlisted 
Aerial  Observers  from  the  same  Squadron  and  Battalion, 
surveyed  on  the  same  dates.  Aerial  Observers  are  enlisted 
soldiers  who  accompany  scout  pilots  during  a  mission. 
Although  their  tasks  do  not  usually  include  hands-on  flying, 
each  has  at  least  60  hours  of  flight  training  prior  to 
reporting  to  an  aviation  unit.  As  flight  crewmembers  working 
in  the  cockpit,  they  have  experience  with  the  surveyed  tasks. 
Thus,  their  responses  contributed  a  third  pool  of  information 
to  the  data  set. 
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TABLE  3.   SURVEY  RESPONDENTS 


2/9  Cavalry 
Sqdn 

1/123 
Attack  Bn 

Total  by 
Group 

Attack  Helicopter 
Pilots 

13 

26 

39 

Scout  Helicopter 
Pilots 

12 

6 

18 

Aerial  Observers 

3 

6 

14 

Total  by  Unit 

33 

38 

71 

2 .   Survey  Administration  Procedures 

The  surveys  were  administered  in  a  classroom  environ- 
ment during  regularly  scheduled  pilot  meetings.  Before 
receiving  the  questionnaire,  the  pilots  and  observers  were 
given  a  short  briefing  by  the  survey  administrator,  an  Army 
captain.  He  used  an  overhead  projector  to  provide  necessary 
information  to  the  subjects  and  emphasized  the  following: 

1.  The  possible  effect  of  their  efforts  on  future  aircraft 
design. 

2.  An  overview  of  the  A3I  and  MIDAS  systems. 

3.  A  short  discussion  of  human  factors  and  its  role  in 
aircraft  design. 

4.  Examples  of  survey  questions  and  responses. 

At  the  end  of  the  briefing,  the  survey  forms  were  distributed 
to  the  respondents  by  each  unit's  instructor  pilots.  All 
forms  were  completed  within  15-25  minutes  after  distribution. 
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III.  ANALYSIS  OF  STUDY  DATA 

A.   DATA  ANALYSIS  SCHEME 

The  data  collected  from  the  71  subjects  surveyed  was 
analyzed  according  to  the  scheme  depicted  in  Figure  2.    The 


Pilot  Background  Data  Box  Plots 
Transform  Raw  Data 


Task  Ratings  Pairwbe  Comparisons 

/ 

Individual  Combined  Individual  Combined 


Scatter  Plots 


Nonparametric  Tests 
(Friedman,  Spearman,  Wilcoxon) 


Correlation  of  Remaining  Survey  Data 


Comparison  of  Survey  Data  With  Model  Data 


Figure  2.   Data  Analysis  Scheme 


concepts  guiding  the  analysis  were: 

1.  Determine   the   degree   of   homogeneity 
respondents. 


among   the 
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2.  Attain  workload  values  for  the  tasks  evaluated  by  the  six 
groups  of  subjects  from  a  Cavalry  Squadron  and  an  Attack 
Helicopter  Battalion,  each  consisting  of  attack 
helicopter  pilots,  scout  helicopter  pilots,  and  aerial 
observers. 

3.  Filter  the  groups  of  data  by  means  of  nonparametric 
statistical  tests. 

4.  Accept  a  final  pool  of  workload  measurements  that  meet 
the  above  criteria. 

These   concepts  were  utilized  to  provide  an   internally 

consistent  set  of  study  data. 

B.   PILOT  BACKGROUND  INFORMATION 

The   last  page  of  the  workload  survey  consisted  of 

questions  intended  to  establish  each  subject's  aviation 

background  and  experience  (see  Appendix  A  or  B)  .   The  pilot 

background  data  was  graphically  analyzed  using  Box  plots 

(Rice,  1988,  pp. 336-337)  in  order  to  compare  the  aviators  in 

the  2/9  Cavalry  Squadron  with  those  of  the  1/12  3  Attack 

Helicopter   Battalion.     Three   particular   measures   were 

analyzed: 

1.  Total  flight  time. 

2.  Flight  time  in  the  aviator's  primary  aircraft:  either  the 
attack  helicopter  or  the  observation  helicopter. 

3.  Flight  time  as  a  pilot-in-command. 

Figure  3  shows  a  multiple  Box  plot  of  the  total  flight 
time  of  all  the  subjects.  The  first  plot  represents  the 
attack  helicopter  pilots  of  the  Attack  Battalion  (AH-AB) 
alongside  those  of  the  Cavalry  Squadron  (AH-CV)  in  the  second 
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plot.  The  third  and  fourth  plots  (SH-AB  and  SH-CV)  are  those 
of  the  Scout  pilots.  The  two  groups  of  observers  (EO-AB  and 
EO-CV)  are  plotted  last. 
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Figure  3.  Multiple   Box   Plots   of   Grouped   Aviator 
Total  Flight  Time 

These  Box  plots  provide  an  efficient  method  of  analysis 
for  several  groups  of  data.  Several  observations  can  be  made, 
based  on  the  complete  set  of  Box  plots  analyzed  during  this 
study.   These  observations  include  the  following: 

1.  When  grouped  within  their  respective  units,  the  attack 
helicopter  pilots  of  the  Cavalry  Squadron  and  Attack 
Battalion  have  very  similar  backgrounds  in  total  flight 
time,  attack  helicopter  flight  time,  and  pilot-in-command 
flight  time. 

2.  There  was  one  attack  helicopter  pilot  outlier,  a  pilot  in 
the  Cavalry  Squadron  with  almost  3000  hours  of  total 
flight  time. 
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3.  The  grouped  scout  helicopter  pilot  data  indicated  that 
subjects  from  the  Cavalry  Squadron  averaged  approximately 
100  hours  more  experience  in  the  three  flight  time 
categories  than  those  from  the  Attack  Battalion. 

4.  Each  scout  helicopter  unit  had  one  outlier  with  slightly 
more  than  2000  hours  total  flight  time. 

5.  Only  total  flight  time  data  was  analyzed  for  the  enlisted 
aerial  observers.  Box  plots  for  observers  from  both 
units  are  almost  identical. 

C.   DATA  TRANSFORMATION  AND  SUMMARIZATION 

The  data  collected  on  the  survey  forms  was  transformed 
into  24  response  vectors  of  either  six  or  nine  values.  Each 
vector  was  derived  from  grouped  subject  responses  for  either 
the  individual  or  combined  task  data  from  the  rating  scales  or 
paired-comparisons.  In  other  words,  these  24  vectors 
represented  responses  from  the  six  aviator  groups  for  the  four 
sections  of  the  survey. 

The  rating  scale  responses  were  transferred  from  each 
survey  form  to  a  spreadsheet.  There  they  were  recorded, 
summed,  and  averaged.  The  resultant  values  are  listed  in 
Table  4,  where  smaller  values  denote  lower  workload  levels. 
The  paired  comparison  responses  were  transferred  from  the 
survey  forms  into  pooled  groups  of  workload  values  by  means  of 
Thurstone's  Method  (Thurstone,  1963,  pp.  67-81).  The 
particular  method  used,  Thurstone's  Case  V  (Dunn-Rankin,  1983, 
pp.  79-82),  resulted  in  the  values  listed  in  Table  5. 
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TABLE  4.  INITIAL  AVERAGE  TASK  RATING  WORKLOAD  VALUES 


INDIVIDUAL  TASKS 

Attack 

Pilots 

Scout  Pilots 

Observers 

Group 
Code 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

Task 
No. 

Attack 
Bn 

Cav 
Sqdn 

Attack 
Bn 

Cav 
Sqdn 

Attack 

Bn 

Cav 
Sqdn 

1 

3.87 

4.19 

2.17 

2.08 

2  .83 

4.19 

2 

3.40 

3.46 

2.50 

2.75 

3.83 

4.69 

3 

2.33 

2.38 

1.17 

1.58 

2.83 

2.19 

4 

2.56 

2.81 

2.17 

1.83 

2.17 

2.81 

5 

1.  13 

0.88 

0.67 

0.83 

0.58 

1.31 

6 

2.67 

3.77 

1.83 

2.25 

2.33 

3.06 

7 

3.29 

3.38 

1.00 

2.42 

1.83 

2.19 

8 

2.06 

2.58 

2.33 

2.00 

1.33 

1.56 

9 

3.42 

2.92 

5.50 

5.92 

4.50 

6.56 

COMBINED  TASKS 

1 

2.33 

4.85 

4.29 

3.58 

4.83 

4.69 

2 

3.83 

4.85 

4.17 

4.75 

5.50 

5.56 

3 

1.33 

2.62 

2.63 

1.42 

3.50 

1.69 

4 

0.50 

1.62 

2.10 

1.33 

1.67 

1.56 

5 

3.17 

4.00 

4.13 

3.08 

4.33 

3.19 

6 

5.50 

4.  35 

4.94 

5.58 

5.83 

5.44 

n 

26 

13 

6 

12 

6 

8 
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TABLE  5.   INITIAL   AVERAGE   PAIRED-COMPARISON   WORKLOAD 
VALUES 


INDIVIDUAL  TASKS 

Attack 

Pilots 

Scout  Pilots 

Observers 

Group 
Code 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

Task 
No. 

Attack 
Bn 

Cav 
Sqdn 

Attack 
Bn 

Cav 
Sqdn 

Attack 
Bn 

Cav 

Sqdn 

1 

1.93 

2.77 

2.90 

1.25 

1.97 

1.47 

2 

1.82 

2.45 

2.90 

1.59 

2.44 

1.66 

3 

1.13 

0.87 

1.40 

0.27 

1.78 

0.55 

4 

1.41 

1.14 

1.91 

0.89 

1.72 

0.57 

5 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

6 

1.59 

1.76 

2.28 

1.19 

1.88 

0.46 

7 

1.68 

2.14 

1.19 

0.93 

0.93 

0.31 

8 

1.23 

1.68 

2.06 

0.91 

1.03 

0.54 

9 

1.54 

1.77 

3.99 

2.82 

2.74 

2.42 

COMBINED  TASKS 

1 

1.51 

1.75 

2.48 

1.87 

1.08 

1.08 

2 

2.08 

2.12 

2.49 

1.95 

2.80 

1.39 

3 

0.53 

0.77 

1.00 

0.30 

0.78 

0.00 

4 

0.00 

0.00 

0.00 

0.00 

0.00 

0.09 

5 

1.39 

2.18 

2.26 

1.72 

0.85 

1.39 

6 

1.75 

2.60 

3.43 

2.33 

2.88 

1.89 

n 

26 

13 

6 

12 

6 

8 
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The  workload  values  listed  in  Tables  4  and  5  are 
characterized  as  follows: 

1.  Group  codes,  identifying  columns  of  workload  values, 
correspond  with  the  similarly-labelled  graphic  data  for 
the  Sguadron  and  Battalion  used  throughout  this  study 
(See  Figure  3) . 

2.  Tasks  are  listed  in  order  of  their  occurrence  during  the 
flight  scenario. 

3.  The  greater  the  workload  value,  the  more  difficult  the 
task. 

4.  Thurstone's  method  assigns  the  easiest  task  the  value 
zero.  Higher-valued  tasks  are  assigned  a  quantitative 
workload  measure  according  to  their  relative  difficulty 
in  relation  to  the  other  tasks. 

5.  The  bottom  row  in  each  table  indicates  the  number  of 
respondents,  n,  whose  responses  were  used  to  calculate 
each  column's  workload  values. 

These  24  vectors  of  workload  data  provide  the  input  for  the 

subsequent  analysis  and  tests  to  follow. 

D.   PLOTS  OF  WORKLOAD  DISTRIBUTION 

Plots  of  the  workload  data  in  Tables  4  and  5  provided  a 
quick  graphic  assessment  of  the  distribution  of  workload 
according  to  task  (Chambers,  1983,  pp.  82-86).  Most  of  the 
plots  constructed  for  the  study  compared  the  responses  of  one 
group  of  aviators  from  the  Cavalry  Squadron  with  those  in  the 
same  type  of  group  from  the  Attack  Helicopter  Battalion. 
Three  representative  graphs  are  depicted  in  Figures  4,  5,  and 
6. 

Figure  4  shows  the  plot  of  individual-task  paired- 
comparison  workload  values  for  the  attack  helicopter  pilots  in 
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both  the  Cavalry  Squadron  and  the  Attack  Helicopter  Battalion. 
Figure  5  shows  combined-task  paired-comparison  workload  data 
for  the  scout  pilots  of  both  units.  Figure  6  shows  a  plot  of 
all  of  the  individual  task  ratings  provided  by  all  subjects, 


4  6 

INDIVIDUAL  TASKS 


Figure  4.  Plot  of  Attack  Helicopter  Pilot  Workload 
Measurements  Based  on  Individual-Task  Paired 
Comparisons 
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Figure  5.  Plot  of  Scout  Helicopter  Pilot  Workload 
Measurements  Based  on  Combined-Task  Paired 
Comparisons 
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averaged  according  to  their  six  respective  aviator  and  unit 
groups.  All  three  plots  indicate  that  there  is  a  general 
trend  toward  like  responses  among  the  groups  of  aviators. 


4  6 

INOIUIDUAL  TASKS 


Figure  6.  Plot  of  Aviator  workload  Measurements  Based  on 
Individual  Task  Ratings  by  All  Study 
Participants 


E.   NONPARAMETRIC  TESTS 

Several  nonparametric  statistical  tests  of  the  data  were 
employed  to  investigate  possible  trends  and  to  determine  if 
responses  were  consistent  among  the  subjects.  These  tests 
were  used  to  explore  the  validity  of  combining  the  workload 
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data  vectors  into  fewer  pools  of  data,  with  more  respondents 
contributing  to  each  pool. 
1.   Friedman  Test 

The  Friedman  test  for  randomized  block  designs 
(Mendenhall,  1990,  pp.  702-704)  was  used  to  determine  if  the 
responses  of  each  pilot  grouping  came  from  identical 
distributions.  The  significance  level  for  this  test  was  set 
at  0.05.  Thus,  if  the  groups  tested  attained  a  p  value 
greater  than  0.05,  then  they  would  be  considered  to  come  from 
the  same  underlying  distribution. 

Friedman  tests  were  conducted  on  data  for  each  of  the 
four  sections  of  the  survey.  Individual-  and  combined-task 
ratings  and  paired  comparisons  for  each  of  the  six  aviator 
groups  were  tested  as  randomized  blocks.  Figure  7  graphically 
portrays  the  Friedman  Test  results  for  one  section  of  the 
survey,  the  individual-task  paired  comparisons.  The  varying 
bar  lengths  indicate  that  the  six  groupings  appear  to  differ 
in  the  average  of  the  ranks  given  to  workload  values.  This  is 
numerically  supported  by  the  p  value  of  0.0001  obtained  from 
the  Friedman  test  for  this  randomized  block. 

Similar  results  were  found  for  initial  average 
workload  values  from  the  other  three  survey  sections.  Table 
6  shows  the  p  values  for  data  from  all  four  sections.  Thus, 
for  each  of  the  four  sections,  it  cannot  be  assumed  that  all 
six  groups  of  respondents  are  from  the  same  underlying 
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distribution.  Therefore,  all  six  response  vectors  should  not 
be  combined  into  one  data  pool  for  any  of  the  four  survey 
sections,  as  none  of  the  p  values  exceeded  0.05. 


AH-AB 
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Figure  7.  Friedman  Test  Average  Ranks  of  Aviator  Groups 
for  the  Individual  Task  Paired-Comparisons 


TABLE  6.  FRIEDMAN  TEST  P  VALUES 


SECTION 

P   VALUE 

IMPLICATION 

Individual  Task  Ratings 

0.0071 

Do  not  pool  data  from 
top  half  TABLE  4 

Combined  Task  Ratings 

0.0114 

Do  not  pool  data  from 
bottom  half  TABLE  4 

Individual  Task  Paired 
Comparisons 

0.0001 

Do  not  pool  data  from 
top  half  TABLE  5 

Combined  Task  Paired 
Comparisons 

0.0448 

Do  not  pool  data  from 
bottom  half  TABLE  5 
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2.   Spearman  Rank  Correlations 

Since  the  Friedman  test  indicated  that  the  data  could 
not  be  combined  into  one  data  pool,  Spearman  rank  correlations 
(Hamburg,  1987,  pp.  576-578)  were  utilized  to  determine  if 
data  from  any  of  the  four  sections  could  be  pooled  for  any  of 
the  six  aviator  groups.  Comparisons  were  made  between  like 
aviators  of  the  Attack  Battalion  and  the  Cavalry  Squadron. 
For  example,  the  responses  of  the  attack  pilots  of  the  Attack 
Helicopter  Battalion  (AH-AB)  were  correlated  with  those  of  the 
Cavalry  Squadron  (AH-CV)  attack  pilots.  The  significance 
level  of  0.006  (shown  in  the  upper  left  portion  of  Table  7) 
indicates  a  high  degree  of  correlation  between  these  two 
groups  of  responses. 

Of  the  groups  of  aviators,  those  that  correlated  with 
a  significance  level  less  than  0.05  were  considered  to  be 
candidates  for  data  pooling.  Table  7  shows  the  significance 
levels  associated  with  the  Spearman  rank  correlations  of  the 
six  aviator  groups  for  the  individual-task  paired  comparisons. 
Only  the  correlations  of  like  groups  (those  performing  similar 
missions)  are  included.  Additionally,  scout  pilot  and 
enlisted  observer  comparisons  were  only  made  when  the  two 
groups  were  in  the  same  unit.  For  instance,  the  responses  of 
the  Cavalry  Squadron  Observers  (EO-CV)  were  correlated  only 
with  Cavalry  Squadron  scout  pilots  (SH-CV) ,  not  with  Attack 
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Battalion  scouts   (SH-AB) .    However,   cross  unit  observer 
comparisons  were  made  (EO-AB  with  EO-CV) . 

The  results  of  the  Spearman  rank  correlations  for 
individual-task  paired  comparisons  are  more  favorable  to  data 
pooling  than  are  results  obtained  for  data  from  the  other 
survey  sections  (significance  levels  for  the  Spearman  rank 
correlations  of  the  other  three  sections  of  the  survey  are 
located  in  Appendix  C)  .  The  bold  numbers  indicate  groups 
whose  correlations  are  statistically  significant  (i.e.,  less 
than  0.05) . 

TABLE  7.   RANK  CORRELATION  SIGNIFICANCE  LEVELS 


INDIVIDUAL-TASK  PAIRED  COMPARISONS 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

AH-AB 

- 

0.006 

- 

- 

- 

- 

AH-CV 

- 

- 

- 

- 

- 

SH-AB 

- 

0.011 

0.009 

- 

SH-CV 

- 

- 

0.048 

EO-AB 

- 

0.013 

EO-CV 

- 

Figure  8  is  a  graph  of  the  Spearman  rank  correlations 
for  the  five  comparisons  of  data  from  the  individual-task 
paired  comparisons  where  correlations  are  statistically 
acceptable  for  pooling  (AH-CV  and  AH-AB,  SH-CV  and  SH-AB,  SH- 
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AB  and  EO-AB,  SH-CV  and  EO-CV,  and  EO-AB  and  EO-CV) .  These 
five  comparisons  correspond  with  the  five  bold  significance 
levels  in  Table  7.  The  Spearman  test  results  (low 
significance  levels  and  high  correlations)  for  these  five 
group  comparisons  indicates  similar  underlying  distributions 
for  each  pair  and  suggests  that  pooling  may  be  appropriate. 


AH-AB  &  AH-CV 
SH-AB  &  SH-CV 
SH-AB  &  EO-AB 
SH-CV  &  EO-CV 
EO-AB  &  EO-CV 


^ 
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Figure  8.  Spearman  Rank  Correlations  for  the  Individual- 
TasX  Paired  Comparisons 


3.   wilcoxon  Signed  Rank  Test 

Before  accepting  the  groupings  of  data  suggested  by 
the  Spearman  test,  a  final  nonparametric  test  was  conducted. 
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The  Wilcoxon  signed  rank  test  (Mendenhall,  1990,  pp.  680-681) 
provided  a  method  of  determining  the  sameness  or  difference  of 
the  aviator  workload  response  distributions.  The  same  aviator 
group  comparisons  made  during  the  Spearman  test  were  made  with 
the  Wilcoxon  test.  The  p  values  generated  by  the  Wilcoxon 
test  are  listed  in  Table  8  for  the  task  ratings  and  in  Table 
9  for  the  paired  comparisons.  P  values  greater  than  a 
significance  level  of  0.05  imply  that  the  underlying 
distributions  are  similar.  Values  in  this  category  are  printed 
in  bold  type. 


TABLE  8.   WILCOXON  SIGNED  RANK  TEST  RESULTS  FOR  THE  TASK 
RATINGS  OF  CORRELATED  GROUPS 


INDIVIDUAL  TASKS 

COMBINED  TASKS 

Compared  Groups 

NO.  of 
Pairs 

p  value 

No.  of 
pairs 

p  value 

AH-AB  &  AH-CV 

9 

0.17 

6 

0.21 

SH-AB  &  SH-CV 

9 

0.16 

6 

0.14 

EO-AB  &  EO-CV 

9 

0.03 

6 

0.06 

SH-AB  &  EO-AB 

9 

0.44 

6 

0.09 

SH-CV  &  EO-CV 

9 

0.02 

6 

0.09 

Tables  9  and  10  show  the  following: 


1.  Workload  values  may  be  pooled  for  both  the  individual- 
and  combined-task  ratings  for  AH-AB  &  AH-CV,  SH-AB  &  SH- 
CV,  and  SH-AB  &  EO-AB. 
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TABLE  9.   WILCOXON  8IGNED  RANK   TEST   RESULTS   FOR   THE 
PAIRED  COMPARISONS  OF  CORRELATED  GROUPS 


INDIVIDUAL  TASKS 

COMBINED  TASKS 

Attack  Bn/Cav  Sqdn 
Samples 

NO.  Of 
Pairs 

p   value 

No.  of 
pairs 

p  value 

AH-AB  &  AH-CV 

9 

0.14 

6 

0.06 

SH-AB  &  SH-CV 

9 

0.01 

6 

0.06 

EO-AB  &  EO-CV 

9 

0.01 

6 

0.28 

SH-AB  &  EO-AB 

9 

0.04 

6 

0.18 

SH-CV  &  EO-CV 

9 

0.11 

6 

0.06 

2.  Workload  values  may  be  pooled  for  only  the  combined-task 
ratings  for  EO-AB  &  EO-CV  and  SH-CV  &  EO-CV. 

3.  Workload  values  may  be  pooled  for  both  the  individual- 
and  combined-task  paired  comparisons  for  AH-AB  &  AH-CV 
and  SH-CV  &  EO-CV. 

4.  Workload  values  may  be  pooled  for  only  the  combined-task 
paired  comparisons  for  SH-AB  &  SH-CV,  EO-AB  &  EO-CV,  and 
SH-AB  &  EO-AB. 

Results  obtained  from  the  Wilcoxon  signed  rank  test  indicate 
that  other  values  cannot  be  pooled.   The  test  showed  that  the 
underlying  distributions  were  not  the  same. 
F.   POOLED  WORKLOAD  RESPONSES 

Pooled  workload  values  resulting  from  the  filtering  effect 
of  the  nonparametric  tests  are  listed  in  Table  10  for  the  task 
ratings  and  Table  11  for  the  paired  comparisons.  The  data  in 
Tables  10  and  11  represents  the  highest  level  of  pooling  that 
can  be  justified  by  the  nonparametric  tests.  In  other  words, 
data  from  the  groups  of  aviators  was  pooled  as  shown  in  Tables 
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10  and  11  if  the  Spearman  test  and  the  Wilcoxon  test  signifi- 
cance level  values  are  shown  in  bold  type  in  both  Table  7  (or 
Appendix  C)  and  Table  8  or  9 . 

The  values  listed  in  Tables  10  and  11  were  generated  by 
recomputing  the  raw  data  from  the  survey  forms.  The 
recomputations  involved  transforming  the  pooled  aviator 
response  data  into  appropriate  task  ratings  and  Thurstone 
paired-comparison  values.  This  process  was  carried  out  only 
for  data  groups  shown  in  Tables  4  and  5  which  were  highly 
correlated  according  to  the  Spearman  test,  and  were  determined 
to  be  of  the  same  underlying  distribution  by  the  Wilcoxon 
test.  Data  that  did  not  meet  these  reguirements  was  not 
included  in  Tables  10  and  11.  Thus,  four  columns  in  Table  10 
and  four  columns  in  Table  11  are  blank. 

Absent  from  Tables  10  and  11  are  pooled  workload  values 
for  the  enlisted  aerial  observers  (EO-AH  and  EO-CV) .  As  these 
aviators  are  not  qualified  pilots,  their  responses  considered 
alone  may  not  accurately  portray  workload.  Therefore,  the 
observers'  responses  were  included  in  Tables  10  and  11  only 
when  they  agreed  with  those  of  the  scout  pilots,  according  to 
the  Spearman  and  Wilcoxon  tests.  The  pooled  workload  values 
of  the  two  right  hand  columns  of  Tables  10  and  11  include 
responses  from  the  enlisted  aerial  observers  pooled  with  those 
of  the  scout  pilots. 
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TABLE  10.  POOLED  AVERAGE  TASK  RATING  WORKLOAD  VALUES 


INDIVIDUAL 

TASKS 

Group 
Code 

AHR 

SHR 

SOR-AB 

SOR-CV 

Task 
No. 

All 
Attack 
Pilots 

All 

Scout 

Pilots 

Attack  Bn 

Scout  & 

Obser 

Cavalry 

Sqdn  Scout 

&  Obser 

1 

3.97 

Cannot  be 

pooled 
according 

to 

Spearman 

Test 

Cannot  be 

pooled 
according 

to 

Spearman 

Test 

Cannot  be 

pooled 
according 

to 

Wilcoxon 

Test 

2 

3.42 

3 

2.35 

4 

2.64 

5 

1.05 

6 

3.04 

7 

3.32 

8 

2.23 

9 

3.26 

COMBINED  1 

\ASKS 

1 

Cannot  be 

pooled 
according 

to 

Spearman 

Test 

3.17 

3.58 

4.03 

2 

4.44 

4.67 

5.08 

3 

1.39 

2.42 

1.52 

4 

1.06 

1.08 

1.43 

5 

3.11 

3.75 

3.13 

6 

5.56 

5.67 

5.53 

n 

39 

18 

12 

20 
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TABLE  11.  POOLED   AVERAGE   PAIRED-COMPARISON   WORKLOAD 
VALUES 


INDIVIDUAL  TASKS 

Group 
Code 

AHP 

SHP 

SOP-AB 

SOP-CV 

Task 
No. 

All 
Attack 
Pilots 

All 

Scout 

Pilots 

Attack  Bn 

Scout  & 

Obser 

Cavalry 

Sqdn  Scout 

&  Obser 

1 

2.03 

Cannot  be  pooled 
according  to 
Wilcoxon 
test 

1.32 

2 

1.87 

1.58 

3 

0.96 

0.41 

4 

1.28 

0.78 

5 

0.00 

0.00 

6 

1.55 

0.94 

7 

1.69 

0.69 

8 

1.28 

0.81 

9 

1.50 

2.64 

COMBINED  TASKS 

1 

Cannot  be 

pooled 
according 

to 

Spearman 

test 

1.99 

0.93 

Cannot  be 

pooled 

according 

to 

Spearman 

test 

2 

1.99 

1.02 

3 

0.50 

0.07 

4 

0.00 

0.00 

5 

1.72 

0.94 

6 

2.51 

1.35 

n 

39 

18 

12 

20 
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G.   CORRELATION  OF  RATINGS  WITH  PAIRED-COMPARISONS 

Spearman's  rank  correlations  were  used  with  the  pooled 
data  of  Tables  10  and  11  to  verify  the  internal  consistency  of 
aviator  responses.  Workload  ratings  for  the  individual  and 
combined  tasks  were  compared  with  the  paired-comparison 
workload  values  for  the  same  tasks  for  those  cases  where  both 
Table  10  and  Table  11  had  data  in  the  same  columns. 

For  the  individual  tasks  the  only  correlation  that  met 
this  requirement  was  that  for  the  attack  helicopter  pilots. 
The  significance  level  of  0.007  for  the  Spearman  correlation 
of  all  attack  helicopter  pilot  individual-task  workload 
ratings  (AHR,  Table  10)  with  related  paired-comparison 
workload  values  (AHP,  Table  11)  indicates  consistent  attack 
helicopter  pilot  responses  throughout  the  survey  for  these 
tasks. 

Combined-task  workload  values  from  Tables  10  and  11  also 
were  tested.  Table  12  provides  the  significance  levels  found 
for  the  Spearman  correlations.  Only  scout  helicopter  pilot 
and  enlisted  aerial  observer  response  data  are  included,  as 
the  attack  helicopter  pilot  responses  could  not  be  pooled. 
All  values  in  Table  12  are  below  the  0.05  significance  level, 
suggesting  consistency  in  the  scout  and  observer  responses  for 
the  combined  tasks. 
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Results  of  the  nonparametric  tests  prevented  pooling  of 
scout  helicopter  pilot  individual  task  responses  for  the 
Attack  Helicopter  Battalion  and  the  Cavalry  Squadron.  Thus 
Spearman  correlations  were  performed  on  the  initial  workload 
values  for  the  individual  tasks  of  the  scout  helicopter  pilots 
(Tables  4  and  5).  Attack  Battalion  scout  pilots'  individual- 
task  ratings  (SH-AB,  Table  4)  and  paired-comparisons  (SH-AB, 
Table  5)  correlated  with  a  significance  level  of  0.013. 
Cavalry  Squadron  scout  pilots'  individual-task  ratings  (SH-CV, 
Table  4)  and  paired  comparisons  (SH-CV,  Table  5)  correlated 
with  a  significance  level  of  0.008.  As  with  the  pooled 
observations  above,  the  significance  levels  for  correlation 
between  the  two  kinds  of  initial  workload  values  indicates 
consistency  among  the  aviators'  responses. 


TABLE  12.  RANK  CORRELATION  SIGNIFICANCE  LEVELS  FOR  POOLED 
WORKLOAD  VALUES 


COMBINED  TASKS: 

RATINGS 

AND  PAIRED  COMPARISONS 

SHR 

SOR-AB 

SOR-CV 

SHP 

SOP-AB 

SHR 

- 

0.035 

0.000 

0.028 

0.035 

SOR-AB 

- 

0.035 

0.045 

0.000 

SOR-CV 

- 

0.028 

0.035 

SHP 

- 

0.045 

SOP-AB 

- 
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Nonpar ametric  test  results  also  prevented  pooling  of 
attack  helicopter  pilot  combined-task  responses  for  the  Attack 
Helicopter  Battalion  and  the  Cavalry  Squadron.  Thus,  as  with 
the  scout  pilot  comparisons  above,  Spearman  correlations  were 
performed  on  the  initial  workload  values  (Tables  4  and  5) . 
Attack  Battalion  attack  pilots'  combined-task  ratings  (AH-AB, 
Table  4)  and  paired  comparisons  (AH-AB,  Table  5)  correlated 
with  a  significance  level  of  0.048.  Cavalry  Squadron  attack 
pilots'  combined-task  ratings  (AH-CV,  Table  4)  and  paired 
comparisons  (AH-CV,  Table  5)  correlated  with  a  significance 
level  of  0.24  3.  From  these  correlations,  the  Attack  Battalion 
attack  pilots'  combined-task  workload  values  are  consistent 
for  the  ratings  and  the  paired  comparisons.  However,  the  high 
significance  level  (0.243)  for  the  Cavalry  Squadron  does  not 
support  consistency  between  the  responses  to  the  ratings  and 
the  paired  comparisons. 

For  the  correlations  of  the  task  rating  workload  values 
with  the  paired-comparison  workload  values,  low  significance 
levels  indicate  high  correlations.  The  results  discussed 
above  show  that  the  subjects'  responses  were  consistent 
throughout  the  survey.  Furthermore,  test  results  confirm 
robustness  of  the  data  and  a  high  degree  of  reliability  among 
the  selected  groups  of  responses.  Based  on  these  tests,  the 
data  in  Tables  10  and  11  and  selected  values  of  Tables  4  and 
5   (except  for  Cavalry  Squadron  attack  helicopter  pilot 
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responses)  can  be  confidently  used  as  baseline  workload  values 
for  the  tasks  and  conditions  considered.  These  baseline, 
empirically-determined  values  serve  as  "ground  truth"  data 
that  can  be  used  to  validate  the  Midas  Task  Loading  Model's 
analytically-produced  values  for  the  same  tasks. 
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IV.  COMPARISON  OF  MIDAS  MODEL  DATA  WITH  STUDY  DATA 

A.   TASK  LOADING  MODEL  WORKLOAD  VALUES 

The  MIDAS  TLM  generates  workload  data  at  a  higher 
resolution  than  was  measured  by  the  survey  forms.  The  TLM 
measures  workload  along  four  dimensions  of  human  performance. 
These  dimensions  are  the  visual,  auditory,  cognitive,  and 
motor  activities  performed  by  an  aviator  while  flying.  In  its 
present  configuration,  the  TLM  averages  workload  values 
generated  along  each  of  the  four  dimensions  for  each  of  the 
individual  and  combined  tasks.  The  tasks  modelled  in  the  TLM 
are  comparable  to  the  nine  individual  tasks  and  six  combined 
tasks  used  in  the  pilot  survey.  The  result  is  a  set  of  task 
workload  values  as  listed  in  Table  13.  The  numbers  represent 
relative  workload  on  a  scale  of  0  to  100,  with  larger  numbers 
indicating  higher  workload.  (Staveland,  1990,  pp. 6-9) 

Two  different  techniques  were  used  with  the  TLM  system  to 
create  workload  values  for  the  scout  and  attack  helicopter 
pilot  tasks.  The  first  technique  involved  measuring  only  the 
workload  generating  attributes  of  the  human  performance 
dimensions  involved  in  a  task  beyond  the  activity  of  flying. 
That  is,  only  workload  related  to  human  performance  above  the 
basic  activity  of  flying  was  calculated.    This  technique 
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generated  the  first  two  columns  of  data  in  Table  13  (ATK  and 
SCT) .  The  second  technique  combined  the  workload  attributes 
of  a  task  with  an  underlying  activity  of  basic  flying.   Thus 

TABLE  13.  TLM-GENERATED  WORKLOAD  VALUES 


INDIVIDUAL  TASKS 

Group 
Code 

ATK 

SCT 

BSCATK 

BSCSCT 

Task 
No. 

Attack 
Pilots 

Scout 
Pilots 

Attack 
Pilots 
(Basic) 

Scout 
Pilots 
(Basic) 

1 

29 

29 

45 

45 

2 

27 

27 

42 

42 

3 

27 

27 

42 

42 

4 

26 

26 

42 

42 

5 

18 

18 

44 

44 

6 

26 

26 

42 

42 

7 

16 

13 

47 

46 

8 

19 

34 

36 

54 

9 

33 

32 

30 

46 

COMBINED  TASKS 

1 

30 

30 

46 

46 

2 

60 

60 

42 

42 

3 

57 

57 

44 

44 

4 

47 

47 

46 

46 

5 

45 

39 

42 

50 

6 

56 

53 

52 

53 
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the  workload  of  each  task  included  the  supposition  of  an 
ongoing  basic  hands-on  flying  activity.  This  technique 
generated  the  last  two  columns  of  data  in  Table  13  (BSCATK  and 
BSCSCT) .   (Staveland,  1990,  pp. 19-23) 

From  a  visual  inspection  of  the  data  in  Table  13,  it 
appears  that  the  TLM-generated  values  for  attack  and  scout 
pilots  are  similar  in  columns  ATK  and  SCT  and  in  columns 
BSCATK  and  BSCSCT,  respectively.  Except  for  tasks  7  and  8, 
the  TLM  values  indicate  that  the  scout  and  attack  helicopter 
pilots  should  experience  almost  identical  levels  of  workload 
for  a  given  task.  An  additional  observation  is  that  many 
tasks  within  a  column  of  data  have  identical  or  near  identical 
workload  values.  For  example,  the  individual-task  portion  of 
column  BSCSCT  contains  four  workload  measurements  of  42,  two 
of  46,  and  one  each  of  44  and  45.  According  to  the  TLM  model 
results,  eight  out  of  nine  tasks  involve  essentially  the  same 
amount  of  workload. 
B.   RANK  CORRELATION  OF  MODEL  DATA  WITH  SURVEY  DATA 

The  TLM  model-generated  workload  values  were  compared  with 
the  survey  workload  values  in  a  procedure  similar  to  that 
described  in  Chapter  III,  Section  G,  where  the  task  ratings 
were  correlated  with  the  paired  comparisons  using  the  Spearman 
rank  correlation  technique.   The  survey  workload  values  used 
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for  this  correlation  are  from  Tables  4,  5,  10,  and  11.   The 
model  data  used  is  all  from  Table  13. 

The  results  of  the  Spearman  rank  correlation  technique  are 
provided  in  Tables  14  to  17.  Each  of  the  first  four  tables 
corresponds  to  one  of  the  four  sections  of  the  survey.  Tables 
18  and  19  show  the  results  of  correlating  the  TLM  with  itself. 
Significance  levels  below  0.05,  printed  in  bold  type,  indicate 
high  correlation. 


TABLE  14.  RANK  CORRELATION  SIGNIFICANCE  LEVELS  OF  MODEL 
VERSUS  STUDY  WORKLOAD  VALUES,  FOR  ATTACK 
HELICOPTER  INDIVIDUAL  TASKS 


TLM 

Group 

Code 

Survey  Group  Code 

AHR 

AHP 

ATK 

0.183 

0.358 

BSCATK 

0.403 

0.408 

TABLE  15.  RANK  CORRELATION  SIGNIFICANCE  LEVELS  OF  MODEL 
VERSUS  STUDY  WORKLOAD  VALUES,  FOR  SCOUT 
HELICOPTER  INDIVIDUAL  TASKS 


TLM 

Group 

code 

Survey  Group  Code 

SH-AB 
(ratings) 

SH-CV 
(ratings) 

SH-AB 
(comparisons) 

SH-CV 
(comparisons) 

SCT 

0.027 

0.521 

0.049 

0.225 

BSCSCT 

0.628 

0.504 

0.852 

0.604 
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The  following  observations  may  be  made  from  the  six  tables: 

1.  No  significant  correlations  were  found  for  the  attack 
helicopter  individual  or  combined  tasks  (Tables  14  and 
16)  nor  the  scout  helicopter  combined  tasks  (Table  17) . 

2.  Two  significant  correlations  were  found  for  the  scout 
helicopter  individual  tasks  (Table  15).  The  TLM's  scout 
workload  values  were  strongly  correlated  with  both  the 
ratings  and  paired-comparison  values  from  the  Attack 
Battalion  scout  pilots'  (SH-AB) .  However,  only  six  scout 
pilots  contributed  to  these  two  categories  of  survey 
data. 


TABLE  16.  RANK  CORRELATION  SIGNIFICANCE  LEVELS  OF  MODEL 
VERSUS  STUDY  WORKLOAD  VALUES,  FOR  ATTACK 
HELICOPTER  COMBINED  TASKS 


TLM 

Group 

Code 

Survey  Group  Code 

AH-AB 
(ratings) 

AH-CV 
(ratings) 

AH-AB 
(comparisons) 

AH-CV 
(comparisons 

ATK 

0.565 

1.000 

0.482 

0.949 

BSCATK 

1.000 

0.841 

0.844 

1.000 

TABLE  17.  RANK  CORRELATION  SIGNIFICANCE  LEVELS  OF  MODEL 
VERSUS  STUDY  WORKLOAD  VALUES,  FOR  SCOUT 
HELICOPTER  COMBINED  TASKS 


TLM 

Group 

Code 

Survey  Group  Code 

SHR 

SHP 

SOR-AB 

SOR-CV 

SOP-AB 

SCT 

0.655 

0.897 

0.565 

0.655 

0.565 

BSCSCT 

0.604 

0.490 

0.476 

0.604 

0.476 
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3.  The  significance  levels  for  TLM-survey  data  correlations 
for  both  the  scout  and  attack  helicopter  combined  tasks 
(Tables  16  and  17)  were  generally  poorer  than  those  for 
the  individual  tasks  (Tables  14  and  15) .  This  indicates 
that  the  workload  results  obtained  via  both  technigues 
for  individual  tasks  correlated  somewhat  better  than  did 
those  for  the  combined  tasks. 

4.  The  significance  level  values  in  Tables  18  and  19 
indicate  that  the  TLM's  two  workload  measurement 
technigues  do  not  correlate  highly  with  each  other. 


TABLE  18.   RANK  CORRELATION  SIGNIFICANCE  LEVELS  OF  TLM 
INDIVIDUAL  TASK  WORKLOAD  VALUES 


TLM 

Group 

Code 

TLM  Group  Code 

BSCATK 

BSCSCT 

ATK 

0.233 

- 

SCT 

- 

0.307 

TABLE  19.   RANK  CORRELATION  SIGNIFICANCE  LEVELS  OF  TLM 
COMBINED  TASK  WORKLOAD  VALUES 


TLM 

Group 

Code 

TLM  Group  Code 

BSCATK 

BSCSCT 

ATK 

0.554 

- 

SCT 

- 

0.218 
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C.   GRAPHIC  ANALYSIS  OF  MODEL  DATA  AND  SURVEY  DATA 
1.   Data  Selection  for  Further  Analysis 

Several  graphic  plots  were  prepared  to  make  further 
comparisons  between  results  from  the  two  workload  measuring 
devices.  All  of  the  survey  data  used  for  the  graphic  analysis 
came  from  the  workload  task  ratings  data,  based  on  a  scale  of 
0  to  10.  With  a  minor  transformation,  these  values  were 
directly  comparable  to  the  TLM  output,  which  was  based  on  a 
scale  of  0  to  100.  Survey  data  used  for  the  graphic 
comparisons  included  AHR  (Table  10,  top)  for  the  attack 
helicopter  individual  tasks,  SH-CV  (Table  4,  top)  for  the 
scout  helicopter  individual  tasks,  AH-AB  (Table  4,  bottom)  for 
the  attack  helicopter  combined  tasks,  and  SHR  (Table  10, 
bottom)  for  the  scout  helicopter  combined  tasks. 

The  four  task-rating  data  vectors  utilized  for  the 
graphic  comparisons  (AHR,  SH-CV,  AH-AB,  and  SHR)  have  been 
shown  to  be  robust  and  internally  valid  for  use  as  baseline 
workload  values  (Chapter  III,  Section  G)  .  An  additional 
criterion  for  the  survey  data  used  for  the  comparisons  was 
that  these  vectors  contained  workload  values  from  the  greatest 
number  of  respondents  for  a  given  survey  section.  For 
example,  the  SH-CV  vector  (Table  4,  top)  was  chosen  over  SH-AB 
(Table  4,  top)  because  SH-CV  represented  responses  from  12 
participants  while  SH-AB  values  were  obtained  from  only  six 
participants. 
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2.   Results  of  Analysis 

The  nine  workload  plots  included  in  this  section  are 
based  on  a  workload  scale  ranging  from  0  to  100.  Smaller 
values  represent  lower  workload  values.  Survey  data  is 
plotted  with  a  solid  line  in  each  graph.  Model-generated  data 
is  plotted  with  a  dashed  line.  Due  to  its  demonstrated 
validity  and  robustness,  the  survey  data  has  been  taken  as 
"ground  truth"  for  these  comparisons. 

The  first  plot,  Figure  9,  shows  the  only  set  of  model- 
generated  (SCT)  and  survey  (SH-AB)  workload  values  that 
correlated  with  a  significance  level  below  0.05.  Distinct 
differences  in  workload  across  the  tasks  may  be  observed.  The 
model  data  generally  tracks  values  from  the  survey  data; 
however,  the  model  data's  "flatness"  implies  that  it  may  have 
poorer  resolution  in  measuring  workload  levels.  Nevertheless, 
considering  the  high  data  correlation,  Figure  9  serves  as  a 
base  from  which  to  view  the  other  graphs. 

The  remaining  eight  graphs  are  characterized  by  the 
following  observations: 

1.  Figure  10  shows  that  the  attack  helicopter  individual- 
task  model  data  (ATK)  agrees  reasonably  well  with  the 
associated  survey  data  (AHR) .  Primary  differences  are 
for  Task  1  (Follow  scout)  and  Task  7  (Listen  to  radio 
transmission) . 
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Figure  9.   Highly  Correlated  Scout  Helicopter  Model  (SCT) 
and  Survey  (SH-AB)  Workload  Values 
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Figure  10.  Attack  Helicopter  Model  (ATK)  and  Survey  (AHR) 
Workload  Values 


2.  Figure  11  indicates  that  the  TLM  BSCATK  data  over- 
estimates workload  and  provides  little  resolution  for 
attack  helicopter  individual  tasks. 
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Figure  11.  Attack  Helicopter  Model  (BSCATK)  and  Survey 
(AHR)  Workload  Values 


Figure  12  is  very  similar  to  Figure  9.  The  survey 
data  of  Figure  12  comes  from  the  Cavalry  Squadron 
scout  pilots  (SH-CV) .  The  survey  data  of  Figure  9  comes 
from  the  Attack  Battalion  scout  pilots  (SH-AB) . 
Comparing  Figure  9  with  Figure  12,  it  may  be  seen  that 
the  model  provides  a  slightly  better  measure  of  Attack 
Battalion  scout  pilot  workload  than  Cavalry  Squadron 
workload.  The  major  discrepancy  is  for  Task  7  (Listen 
to  radio   transmission) . 
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Figure  12.  Scout  Helicopter  Model  (SCT)  and  Survey  (SH- 
CV)  Workload  Values 
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4.  Figure  13  is  similar  to  Figure  11.  Again,  the  BSCSCT 
data  overestimates  workload  and  provides  little 
resolution  for  scout  helicopter  individual  tasks. 
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Figure  13.  Scout  Helicopter  Model  (BSCSCT)  and  Survey 
(SH-CV)  Workload  Values 


Figures  14  through  17  all  involve  combined  tasks  and 
display  similar  results.  In  each  case  the  model- 
generated  workload  values  are  much  higher  than  the  survey 
values  for  most  tasks.  The  model  values  for  ATK  (Figure 
14)  and  SCT  (Figure  16)  show  some  trend  agreement  with 
the  survey  values,  but  greatly  overestimate  all  but  the 
first  and  last  tasks.  The  model's  BSCATK  (Figure  15)  and 
BSCSCT  (Figure  17)  values  show  almost  no  resolution  of 
workload  levels  over  the  six  combined  tasks. 
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Figure  14.  Attack  Helicopter  Model  (ATK)  and  Survey  (AH- 
AB)  Workload  Values 
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Figure  15.  Attack  Helicopter  Model  (BSCATK)  and  Survey 
(AH-AB)  Workload  Values 
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Figure  16.  Scout  Helicopter  Model  (SCT)  and  Survey  (SHR) 
Workload  Values 
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Figure  17.  Scout  Helicopter  Model  (BSCSCT)  and  Survey 
(SHR)  Workload  Values 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 

A.   CONCLUSIONS 

The  purpose  of  this  study  was  to  determine  the  validity  of 
the  A3I  TLM  as  a  device  to  measure  the  workload  levels 
experienced  by  Army  helicopter  pilots.  Empirically-derived 
workload  data  was  collected  from  Army  pilots,  analyzed,  and 
correlated  with  workload  measurements  generated  by  the  TLM. 
Analysis  of  the  empirical  data  provided  insights  into  pilot- 
generated  workload  levels  and  provided  a  statistically 
acceptable  basis  upon  which  to  judge  the  accuracy  of  TLM 
outputs.  Furthermore,  insights  into  the  opinions  of  Army 
helicopter  pilots  concerning  workload  provides  direction  for 
future  refinement  of  the  TLM. 

1.   TLM  Validity 

In  its  present  configuration,  output  from  the  TLM  does 
not  consistently  yield  measurements  of  Army  helicopter  pilot 
workload  that  correlate  with  pilot  opinions  of  their  workload 
levels,  for  the  set  of  tasks  tested.  Of  the  two  TLM 
technigues  of  evaluating  workload,  the  second  technigue  should 
be  abandoned.  This  technigue  adds  task-generated  workload 
onto  constant  workload  resulting  from  a  basic  underlying 
flying  task  (BSCATK  and  BSCSCT) .  This  technigue  showed  little 
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workload  level  resolution  and  overestimated  workload  values 
for  almost  every  task. 

However,  the  first  technique  utilized  by  the  TLM 
appears  to  provide  better  results.  This  technique  analyzes 
aviator  tasks  separately  from  the  actions  associated  with 
hands-on  flying.  Graphic  analysis  showed  workload  trends  that 
in  part  agreed  with  the  empirical  measurements  of  workload. 
However,  it  should  be  noted  that  agreement  between  the  model 
and  the  survey  data  was  greater  for  the  individual  task 
measurements  than  for  those  related  to  the  combined  tasks. 
2.   Army  Helicopter  Pilot  Workload  Perceptions 

Outputs  from  the  TLM  currently  indicate  that  workload 
levels  for  the  scout  and  attack  helicopter  pilots  are  almost 
identical.  For  essentially  the  same  set  of  tasks  embedded  in 
realistic  scout  and  attack  helicopter  scenarios,  the  data 
collected  via  the  pilot  surveys  indicates  that  aviator 
workload  varies  according  to  the  type  aircraft  flown  and  the 
type  of  mission  conducted  by  the  pilots.  It  is  obvious  that 
there  are  major  discrepancies  between  the  two  methods  for 
determining  workload. 

Results  of  the  Friedman  test  indicate  that  survey 
responses  from  the  Attack  Helicopter  Battalion  participants 
cannot  be  pooled  with  those  from  the  Cavalry  Squadron.   This 
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is  a  strong  indication  that  workload  levels  are  not 
experienced  the  same  by  all  of  the  aviators  tested.  The 
Spearman  and  Wilcoxon  tests  showed  that,  even  among  the  scout 
pilots,  workload  is  perceived  differently  by  those  scouts  in 
the  Cavalry  Sguadron  than  by  those  in  the  Attack  Helicopter 
Battalion.  These  results  provide  strong  evidence  that  the  TLM 
may  need  modification  to  adjust  workload  according  to  the 
aviator's  aircraft  and  specific  mission. 
B.   RECOMMENDATIONS 

Analysis  of  the  degree  of  correlation  between  the  TLM's 
workload  output  and  the  empirically-generated  workload  values 
obtained  via  the  survey  results  in  three  recommendations. 
First,  the  TLM  in  its  present  configuration  should  not  be  used 
as  the  workload  measuring  mechanism  in  the  MIDAS  program.  The 
TLM  generally  overestimates  workload  and  provides  an  overly- 
narrow  range  of  values  for  workload  experienced  by  pilots 
performing  a  variety  of  tasks. 

Second,  the  TLM  shows  promising  potential  in  the  measuring 
of  workload  associated  with  individual  helicopter  pilot  tasks. 
This  aspect  of  the  TLM  should  be  enhanced  to  take  maximum 
advantage  of  this  capability.  Initial  enhancements  should 
address  the  mission  and  aircraft  involved  in  generating  pilot 
workload.  Although  the  survey  data  indicates  that  these 
factors  are  significant,  the  TLM  generates  generic  workload 
measurements  independent  of  them. 
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Third,  empirical  workload  data  collected  as  described  in 
this  thesis  should  be  used  for  future  evaluations  of  the  TLM. 
As  the  TLM  matures,  efforts  should  be  made  to  ensure  that 
output  correlates  highly  with  workload  data  obtained  directly 
from  Army  pilots,  as  described  in  this  study.  However,  the 
limited  range  of  the  survey  data  reported  here  should  not 
serve  as  the  only  means  of  validating  the  TLM.  TLM  output 
should  also  be  compared  to  other  empirically-derived  workload 
measurements,  and  to  the  output  of  other  models  designed  to 
measure  pilot  workload. 
C.   SUGGESTIONS  FOR  FURTHER  RESEARCH 

Several  aspects  of  this  study  deserve  further  invest- 
igation. First,  helicopter  pilot  workload  data  should  be 
obtained  from  another,  independent  source.  This  data  then 
could  be  compared  with  the  empirically-derived  data  collected 
in  this  study,  to  insure  data  validity.  This  same  independent 
workload  data  also  could  be  compared  to  the  TLM's  output. 
Second,  the  survey  results  reported  here  may  be  based  on 
factors  not  investigated  in  this  study,  such  as  individual 
pilot  experience.  A  more  detailed  analysis  of  the  individual 
pilot  responses  may  be  warranted.  Third,  expanded  surveys 
similar  to  those  used  for  this  study  should  be  conducted  in 
the  future  to  broaden  and  refine  the  workload  data  base. 
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APPENDIX  A.  SCOUT  HELICOPTER  WORKLOAD  SURVEY 


SURVEY  ON  HELICOPTER  PILOTING  WORKLOAD 

Thank  you  for  your  participation  in  this  survey.  Your 
assistance  helps  a  NASA-Army  team  develop  a  human  factors 
design  tool  aimed  at  improving  the  layout  of  future  Army 
helicopter  cockpits. 

There  are  no  right  or  wrong  answers  to  the  questions  in 
this  survey.  The  intent  is  to  determine  relative  workload 
that  you  experience  when  flying.  Your  responses  will  help  us 
determine  what  tasks  you  feel  are  harder  or  easier  than  others 
for  the  given  scenario.  Please  answer  the  questions  candidly. 

The  survey  is  anonymous.  The  combined  data  from  all  of 
the  surveys  will  produce  an  overall  "pilot"  profile  of  task 
ratings. 

The  ratings  and  comparisons  in  the  survey  are  listed 
randomly.   No  two  surveys  are  exactly  alike. 

If  you  have  any  questions,  hold  up  your  hand  and  the 
survey  administrating  officer  will  assist  you. 
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HOW  TO  FILL  OUT  THE  SURVEY 


Evaluate  the  following  maneuvers  as  an  individual  crewman 
in  the  context  of  the  given  scenario  and  helicopter  terrain 
flight  in  general.  Difficulty  is  defined  as  a  relative  measure 
of  combined  physical  and  mental  effort. 

Evaluations  are  done  in  two  parts.  First,  rate  each 
maneuver  by  circling  a  value  along  the  degree-of-dif f iculty 
continuum  to  indicate  its  difficulty. 


Example  1: 

MANEUVER  DEGREE  OF  DIFFICULTY 


low  high 


Fly  straight  and  level     0 1 2 3 4 5 6 7 8 9 10 


Autorotate  at  night  0123456789      10 


For  the  second  part  of  the  survey,  individual  maneuvers 
are  compared  with  each  other. 

Choose  the  more  difficult  of  the  two  by  filling  in  the 
circle  next  to  it. 


Example  2 : 


Fly  straight  and  level   O  O     Autorotate  at  night 
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SCOUT  HELICOPTER  SCENARIO 


SETTING:  You  are  the  pilot  of  an  OH-58  conducting  a  route 
reconnaissance  in  an  unfamiliar  hilly/wooded  area.  You  are  in 
the  terrain  flight  mode  along  one  side  of  the  route,  clearing 
the  area  while  observing  the  route  as  freguently  as  possible. 
Due  to  pilot  shortages,  you  are  accompanied  by  a  crewchief 
sitting  in  the  left  seat.  He  is  familiar  with  the  cockpit. 
He  can  hold  a  1:50,000  map  for  you  to  observe.  However,  he 
cannot  fly  and  you  must  double  check  his  map  reading 
activities.  You  are  flying  during  the  day  with  clear  and  calm 
weather  conditions.  An  attack  helicopter  is  overwatching  you. 
However,  you  will  not  communicate  with  him  during  this  mission 
segment.   Enemy  contact  is  possible. 
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Rate  the  difficulty  of  each  maneuver  by  circling  a  value  along 
the  degree  of  difficulty  continuum. 


MANEUVER  DEGREE  OF   DIFFICULTY 


low  high 


Follow  another  scout  0123456789      10 


NOEdecelerationO      123456789      10 


Check  torque  meter  and  announce  reading    0123456789      10 


Hover       0123456789      10 


Tell  crewchief  "Unmasking"  0 1      23456789      10 


Unmaskaircraft 0      123456789      10 


Listen  to  crewchief  describe  what  he  sees  0 1 2 3 4 5 6 7 8 9 10 

Turn  up  volume  on  ICS  control  panel  0   1 2   3456789   10 


Determine  present  location  (6  digit  grid)  0 1 2 3 4 5 6 7 8 9 10 

as  crewchief  holds  map 
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Rate  the  difficulty  of  each  maneuver  by  circling  a  value  along 
the  degree  of  difficulty  continuum. 

MANEUVER  DEGREE  OF   DIFFICULTY 

low  high 

NOE  deceleration  behind  another  scout  0123456789   10 


Check  torque  meter  and  announce  reading  0 1 2 3 4 5 6 7 8 9 10 

while  NOE  decelerating 


Check  torque  meter  and  announce  reading  0 1 2 3 4 5 6 7 8 9 10 

while  hovering 


Tell  crewchief  "Unmasking"  while  unmasking  0 1 2 3 4 5 6 7 8 9 10 

aircraft 


Listen  to  crewchief  describe  what  he  sees  0 1 2 3 4 5 6 7 8 9 10 

while  you  turn  up  volume  on  ICS  control 

panel  and  hover 


Determine  present  location  (6  digit  grid)  0 1 2 3 4 5 6 7 8 9 10 

as  crewchief  holds  map  and  you  listen 
to  him  describe  what  he  sees 
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Compare  the  two  maneuvers  on  the  same  line.   Choose  the  more 
difficult  of  the  two  by  filling  in  the  circle  next  to  it. 

MANEUVER  1  MANEUVER  2 


Follow  another  scout  O  O  NOE  deceleration 


Follow  another  scout  O  O  Check  torque  meter  and  announce 

reading 


Follow  another  scout  O  O  Hover 


Follow  another  scout  O  O  Tell  crewchief  "Unmasking' 


Follow  another  scout  0  0  Unmask  aircraft 


Follow  another  scout  O  O  Listen  to  crewchief  describe  what 

he  sees 


Follow  another  scout  0  0  Turn  up  volume  on  ICS  control  panel 


Follow  another  scout  O  O  Determine  present  location  (6  digit 

grid)  as  crewchief  holds  map 


NOE  deceleration  O  O  Check  torque  meter  and  announce 

reading 


NOE  deceleration  O  0  Hover 

NOE  deceleration  O  O  Tell  crewchief  "Unmasking" 

NOE  deceleration  O  O  Unmask  aircraft 
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MANEUVER  1  MANEUVER  2 


NOE  deceleration  O  O  Listen  to  crewchief  describe  what 

he  sees 


NOE  deceleration  O  O  Turn  up  volume  on  ICS  control  panel 


NOE  deceleration  O  O  Determine  present  location  (6  digit 

grid)  as  crewchief  holds  map 


Check  torque  meter  and  announce  O  O  Hover 

reading 


Check  torque  meter  and  announce  O  O  Tell  crewchief  "Unmasking" 

reading 


Check  torque  meter  and  announce  O  O  Unmask  aircraft 

reading 


Check  torque  meter  and  announce  O  O  Listen  to  crewchief  describe  what 

reading      he  sees 


Check  torque  meter  and  announce  O  O  Turn  up  volume  on  ICS  control  panel 

reading 


Check  torque  meter  and  announce  O  O  Determine  present  location  (6  digit 

reading      grid)  as  crewchief  holds  map 


Hover  O  O  Tell  crewchief  "Unmasking" 


Hover  O  O  Unmask  aircraft 
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MANEUVER  1  MANEUVER  2 


Hover  0  O  Listen  to  crewchief  describe  what 
he  sees 


Hover  0  O  Turn  up  volume  on  ICS  control  panel 


Hover  O  0  Determine  present  location  (6  digit 
grid)  as  crewchief  holds  map 


Tell  crewchief  "Unmasking"  O  O  Unmask  aircraft 


Tell  crewchief  "Unmasking"  O  O  Listen  to  crewchief  describe  what 

he  sees 


Tell  crewchief  "Unmasking"  O  O  Turn  up  volume  on  ICS  control  panel 


Tell  crewchief  "Unmasking"  O  O  Determine  present  location  (6  digit 

grid)  as  crewchief  holds  map 


Unmask  aircraft  O  O  Listen  to  crewchief  describe  what 

he  sees 


Unmask  aircraft  O  O  Turn  up  volume  on  ICS  control  panel 


Unmask  aircraft  O  O  Determine  present  location  (6  digit 

grid)  as  crewchief  holds  map 


Listen  to  crewchief  describe  what  O  O  Turn  up  volume  on  ICS  control  panel 

he  sees 
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MANEUVER  1  MANEUVER  2 


Listen  to  crewchief  describe  what  O  O  Determine  present  location  (6  digit 

he  sees      grid)  as  crewchief  holds  map 


Turn  up  volume  on  ICS  control  panel  O  O  Determine  present  location  (6  digit 

grid)  as  crewchief  holds  map 
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Compare  the  two  maneuvers  on  the  same  line.   Choose  the  more 
difficult  of  the  two  by  filling  in  the  circle  next  to  it. 

MANEUVER  1  MANEUVER  2 


NOE  deceleration  behind  another  O  O  Check  torque  meter  and  announce 

scout      reading  while  NOE  decelerating 


NOE  deceleration  behind  another  O  O  Check  torque  meter  and  announce 

scout      reading  while  hovering 


NOE  deceleration  behind  another  O  O  Tell  crewchief  "Unmasking"  while 

scout      unmasking  aircraft 


NOE  deceleration  behind  another  0  O  Listen  to  crewchief  describe  what 

scout      he  sees  while  you  turn  up  volume 
on  ICS  control  panel  and  hover 


NOE  deceleration  behind  another  O  0  Determine  present  location  (6  digit 

scout      grid)  as  crewchief  holds  map  and 
you  listen  to  him  describe  what 
he  sees 


Check  torque  meter  and  announce  O  O  Check  torque  meter  and  announce 
reading  while  NOE  decelerating      reading  while  hovering 


Check  torque  meter  and  announce  0  0  Tell  crewchief  "Unmasking"  while 
reading  while  NOE  decelerating      unmasking  aircraft 


Check  torque  meter  and  announce  O  O  Listen  to  crewchief  describe  what 
reading  while  NOE  decelerating      he  sees  while  you  turn  up  volume 

on  ICS  control  panel  and  hover 


SURVEY  #  OHCPC  1-1  PAGE  10  OF  12 

68 


MANEUVER  1 


MANEUVER  2 


Check  torque  meter  and  announce  O  O  Determine  present  location  (6  digit 
reading  while  NOE  decelerating      grid)  as  crewchief  holds  map  and 

you  listen  to  him  describe  what 
he  sees 


Check  torque  meter  and  announce  0 
reading  while  hovering 


0  Tell  crewchief  "Unmasking"  while 
unmasking  aircraft 


Check  torque  meter  and  announce  0 
reading  while  hovering 


O  Listen  to  crewchief  describe  what 
he  sees  while  you  turn  up  volume 
on  ICS  control  panel  and  hover 


Check  torque  meter  and  announce  O  O  Determine  present  location  (6  digit 
reading  while  hovering      grid)  as  crewchief  holds  map  and 

you  listen  to  him  describe  what 
he  sees 


Tell  crewchief  "Unmasking"  while  O  O  Listen  to  crewchief  describe  what 
unmasking  aircraft      he  sees  while  you  turn  up  volume 

on  ICS  control  panel  and  hover 


Tell  crewchief  "Unmasking"  while  0  O  Determine  present  location  (6  digit 
unmasking  aircraft      grid)  as  crewchief  holds  map  and 

you  listen  to  him  describe  what 
he  sees 


Listen  to  crewchief  describe  what  0  O  Determine  present  location  (6  digit 
he  sees  while  you  turn  up  volume      grid)  as  crewchief  holds  map  and 
on  ICS  control  panel  and  hover      you  listen  to  him  describe  what 

he  sees 
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PILOT  BACKGROUND  INFORMATION 

Please  answer  the  following  questions  regarding  your 
aviation  experience  and  qualifications. 

1.    Approximately  how  many  flight  hours  do  you  have  in 
the  following  aircraft  as  either  a  PC  or  PI? 
(Leave  blank  if  little  or  no  experience) 

OH-58A/C  AH-1     

OH-58D  AH-64 


UH-1     other  rotary  wing 

UH-60     fixed  wing     


Answer  the  following  by  indicating  the  approximate 
number  of  flight  hours. 

a)  Your  total  flight  time:  


b)  Flight  time  as  an  Instructor  Pilot: 

c)  Combat  flight  time:  

d)  Flight  time  while  qualified  as  a  PC: 


3.    Answer  the  following  by  indicating  the  approximate 
number  of  years. 

a)  Years  as  a  PC:  


b)  Years  as  an  Army  Aviator: 


END  OF  SURVEY 
THANK  YOU  FOR  YOUR  TIME  AND  EFFORT 
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APPENDIX  B.  ATTACK  HELICOPTER  WORKLOAD  SURVEY 

SURVEY  ON  HELICOPTER  PILOTING  WORKLOAD 

Thank  you  for  your  participation  in  this  survey.  Your 
assistance  helps  a  NASA-Army  team  develop  a  human  factors 
design  tool  aimed  at  improving  the  layout  of  future  Army 
helicopter  cockpits. 

There  are  no  right  or  wrong  answers  to  the  questions  in 
this  survey.  The  intent  is  to  determine  relative  workload 
that  you  experience  when  flying.  Your  responses  will  help  us 
determine  what  tasks  you  feel  are  harder  or  easier  than  others 
for  the  given  scenario.  Please  answer  the  questions  candidly. 

The  survey  is  anonymous.  The  combined  data  from  all  of 
the  surveys  will  produce  an  overall  "pilot"  profile  of  task 
ratings. 

The  ratings  and  comparisons  in  the  survey  are  listed 
randomly.   No  two  surveys  are  exactly  alike. 

If  you  have  any  questions,  hold  up  your  hand  and  the 
survey  administrating  officer  will  assist  you. 
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HOW  TO  FILL  OUT  THE  SURVEY 


Evaluate  the  following  maneuvers  as  an  individual  crewman 
in  the  context  of  the  given  scenario  and  helicopter  terrain 
flight  in  general.  Difficulty  is  defined  as  a  relative  measure 
of  combined  physical  and  mental  effort. 

Evaluations  are  done  in  two  parts.  First,  rate  each 
maneuver  by  circling  a  value  along  the  degree-of-dif f iculty 
continuum  to  indicate  its  difficulty. 


Example  1: 

MANEUVER  DEGREE  OF  DIFFICULTY 


low  high 


Fly  straight  and  level     0   1   2 3   4   5   6   7   8   9   10 


Autorotate  at  night  0123456789       10 


For  the  second  part  of  the  survey,  individual  maneuvers 
are  compared  with  each  other. 

Choose  the  more  difficult  of  the  two  by  filling  in  the 
circle  next  to  it. 


Example  2: 


Fly  straight  and  level   0  0     Autorotate  at  night 
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ATTACK  HELICOPTER  SCENARIO 

SETTING:  You  are  the  pilot  of  an  armed  AH-1  flying  in  the 
terrain  flight  mode.  You  are  conducting  travelling  overwatch 
by  following  an  aeroscout  through  an  unfamiliar  hilly/wooded 
area  to  a  battle  position.  You  will  receive  a  target  handoff 
from  the  firing  position.  Your  crewmember  will  prepare  for  a 
TOW  missile  engagement  and  will  not  fly  the  aircraft  during 
this  mission  segment.  You  are  flying  during  the  day  with 
clear  and  calm  weather  conditions.  Enemy  contact  is  possible. 
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Rate  the  difficulty  of  each  maneuver  by  circling  a  value  along 
the  degree  of  difficulty  continuum. 

MANEUVER  DEGREE  OF   DIFFICULTY 


low  high 


FollowscoutO      123456789      10 


NOEdecelerationO      123456789      10 


Check  torque  meter  and  announce  reading  0123456789   10 


Hover       0123456789      10 


Tell  crewmember  "Unmasking"  0123456789      10 


Unmaskaircraft 0      123456789      10 


Listen  to  radio  transmission  from  scout  0 1 2 3 4 5 6 7 8 9 10 

(direction  to  fire,  range,  target) 


Switch  master  arm  to  "arm"    0123456789      10 


Interpret  gunners  position  of  TSU  based  on  0 1 2 3 4 5 6 7 8 9 10 

PSI  information 
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Rate  the  difficulty  of  each  maneuver  by  circling  a  value  along 
the  degree  of  difficulty  continuum. 

MANEUVER  DEGREE  OF   DIFFICULTY 

low  high 

NOE  deceleration  behind  scout  0123456789   10 


Check  torque  meter  and  announce  reading  0 1 2 3 4 5 6 7 8 9 10 

while  NOE  decelerating 


Check  torque  meter  and  announce  reading  0 1 2 3 4 5 6 7 8 9 10 

while  hovering 


Tell  crewmember  "Unmasking"  while  unmasking  0 1 2 3 4 5 6 7 8 9 10 

aircraft 


Listen  to  radio  transmission  from  scout  0 1 2 3 4 5 6 7 8 9 10 

(direction  to  fire,  range,  target)  while 
switchinq  master  arm  to  "arm" 


Interpret  gunners  position  of  TSU,  based  0 1 2 3 4 5 6 7 8 9 10 

on  PSI  information,  while  receiving 
target  handoff  from  scout 
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Compare  the  two  maneuvers  on  the  same  line.   Choose  the  more 
difficult  of  the  two  by  filling  in  the  circle  next  to  it. 

MANEUVER  1  MANEUVER  2 


Follow  scout  O   O  NOE  deceleration 


Follow  scout  O  O  Check  torque  meter  and  announce 
reading 


Follow  scout  O   O  Hover 


Follow  scout  O  O  Tell  crewmember  "Unmasking' 


Follow  scout  0  O  Unmask  aircraft 


Follow  scout  O  O  Listen  to  radio  transmission  from 
scout  (target,  range,  direction) 


Follow  scout  0  0  Switch  master  arm  to  "arm" 


Follow  scout  0  0  Interpret  gunners  position  of  TSU 
based  on  PSI  information 


NOE  deceleration  O  O  Check  torque  meter  and  announce 

reading 


NOE  deceleration  O  0  Hover 


NOE  deceleration  0  0  Tell  crewmember  "Unmasking" 
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MANEUVER  1  MANEUVER  2 


NOE  deceleration  O  O  Unmask  aircraft 


NOE  deceleration  O  O  Listen  to  radio  transmission  from 

scout  (target,  range,  direction) 


NOE  deceleration  O  O  Switch  master  arm  to  "arm" 


NOE  deceleration  0  0  Interpret  gunners  position  of  TSU 

based  on  PSI  information 


Check  torque  meter  and  announce  0  O  Hover 

reading 


Check  torque  meter  and  announce  0  0  Tell  crewmember  "Unmasking' 

reading 


Check  torque  meter  and  announce  O  O  Unmask  aircraft 

reading 


Check  torque  meter  and  announce  0  O  Listen  to  radio  transmission  from 

reading     scout  (target,  range,  direction) 


Check  torque  meter  and  announce  O  0  Switch  master  arm  to  "arm" 

reading 


Check  torque  meter  and  announce  0  0  Interpret  gunners  position  of  TSU 

reading      based  on  PSI  information 


Hover  0  0  Tell  crewmember  "Unmasking" 
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MANEUVER  1  MANEUVER  2 


Hover  O  O  Unmask  aircraft 


Hover  O  O  Listen  to  radio  transmission  from 
scout  (target,  range, direction) 


Hover  O  O  Switch  master  arm  to  "arm" 


Hover  0  O  Interpret  gunners  position  of  TSU 
based  on  PSI  information 


Tell  crewmember  "Unmasking"  O  O  Unmask  aircraft 


Tell  crewmember  "Unmasking"  O  0  Listen  to  radio  transmission  from 

scout  (target,  range,  direction) 


Tell  crewmember  "Unmasking"  O  0  Switch  master  arm  to  "arm" 


Tell  crewmember  "Unmasking"  0  0  Interpret  gunners  position  of  TSU 

based  on  PSI  information 


Unmask  aircraft  O  O  Listen  to  radio  transmission  from 

scout  (target,  range,  direction) 


Unmask  aircraft  0  O  Switch  master  arm  to  "arm' 


Unmask  aircraft  O  O  Interpret  gunners  position  of  TSU 

based  on  PSI  information 


Listen  to  radio  transmission  from  0  O  Switch  master  arm  to  "arm" 
scout  (target,  range,  direction) 
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MANEUVER  1 


MANEUVER  2 


Listen  to  radio  transmission  from  O 
scout  (target,  range,  direction) 


Interpret  gunners  position  of  TSU 
based  on  PSI  information 


Switch  master  arm  to  "arm"  O 


O  Interpret  gunners  position  of  TSU 
based  on  PSI  information 
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Compare  the  two  maneuvers  on  the  same  line.   Choose  the  more 
difficult  of  the  two  by  filling  in  the  circle  next  to  it. 

MANEUVER  1  MANEUVER  2 


NOE  deceleration  behind  scout  O  0  Check  torque  meter  and  announce 

reading  while  NOE  decelerating 


NOE  deceleration  behind  scout  O  0  Check  torque  meter  and  announce 

reading  while  hovering 


NOE  deceleration  behind  scout  O  O  Tell  crewmember  "Unmasking"  while 

unmasking  aircraft 


NOE  deceleration  behind  scout  0  0  Listen  to  radio  xsmn  from  scout 

(direction,  range,  target)  while 
switching  master  arm  to  "arm" 


NOE  deceleration  behind  scout  O  O  Interpret  gunners  position  of  TSU 

based  on  PSI  info,  while 
receiving  target  handoff  from 
scout 


Check  torque  meter  and  announce  O  0  Check  torque  meter  and  announce 
reading  while  NOE  decelerating      reading  while  hovering 


Check  torque  meter  and  announce  O  O  Tell  crewmember  "Unmasking"  while 
reading  while  NOE  decelerating      unmasking  aircraft 


Check  torque  meter  and  announce  O  O  Listen  to  radio  xsmn  from  scout 
reading  while  NOE  decelerating      (direction,  range,  target)  while 

switching  master  arm  to  "arm" 


Check  torque  meter  and  announce  O  O  Interpret  gunners  position  of  TSU 
reading  while  NOE  decelerating      based  on  PSI  info,  while 

receiving  target  handoff  from 
scout 
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MANEUVER  1 


MANEUVER  2 


Check  torque  meter  and  announce  O  O  Tell  crewmember  "Unmasking"  while 
reading  while  hovering      unmasking  aircraft 


Check  torque  meter  and  announce  O  O  Listen  to  radio  xsmn  from  scout 
reading  while  hovering     (direction,  range,  target)  while 

switching  master  arm  to  "arm" 


Check  torque  meter  and  announce  0  0  Interpret  gunners  position  of  TSU 
reading  while  hovering      based  on  PSI  info,  while 

receiving  target  handoff  from 
scout 


Tell  crewmember  "Unmasking"  while  0  O  Listen  to  radio  xsmn  from  scout 
unmasking  aircraft      (direction,  range,  target)  while 

switching  master  arm  to  "arm" 


Tell  crewmember  "Unmasking"  while  0  O  Interpret  gunners  position  of  TSU 
unmasking  aircraft      based  on  PSI  info,  while 

receiving  target  handoff  from 
scout 


Listen  to  radio  xsmn  from  scout  0  0  Interpret  gunners  position  of  TSU 
(direction,  range,  target)  while      based  on  PSI  info,  while 

switching  master  arm  to  "arm"       receiving  target  handoff  from 

scout 
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PILOT  BACKGROUND  INFORMATION 

Please  answer  the  following  questions  regarding  your 
aviation  experience  and  qualifications. 

1.    Approximately  how  many  flight  hours  do  you  have  in 
the  following  aircraft  as  either  a  PC  or  PI? 
(Leave  blank  if  little  or  no  experience) 

OH-58A/C  AH-1     

OH-58D  AH-64 


UH-1     other  rotary  wing 

UH-60    fixed  wing    


Answer  the  following  by  indicating  the  approximate 
number  of  flight  hours. 

a)  Your  total  flight  time:  


b)  Flight  time  as  an  Instructor  Pilot: 

c)  Combat  flight  time:  

d)  Flight  time  while  qualified  as  a  PC: 


3.    Answer  the  following  by  indicating  the  approximate 
number  of  years. 

a)  Years  as  a  PC:  


b)  Years  as  an  Army  Aviator: 


END  OF  SURVEY 
THANK  YOU  FOR  YOUR  TIME  AND  EFFORT 
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APPENDIX  C.  SPEARMAN  RANK  CORRELATION  SIGNIFICANCE  LEVELS 


INDIVIDUAL-TASK  PAIRED  COMPARISONS 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

AH-AB 

- 

0.006 

- 

- 

- 

- 

AH-CV 

- 

- 

- 

- 

- 

SH-AB 

- 

0.011 

0.009 

- 

SH-CV 

- 

- 

0.048 

EO-AB 

- 

0.013 

EO-CV 

- 

INDIVIDUAL-TASK  RATINGS 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

AH-AB 

- 

0.018 

- 

- 

- 

- 

AH-CV 

- 

- 

- 

- 

- 

SH-AB 

- 

0.079 

0.069 

- 

SH-CV 

- 

- 

0.027 

EO-AB 

- 

0.011 

EO-CV 

- 
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COMBINED-TASK  PAIRED  COMPARISONS 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

AH-AB 

- 

0.110 

- 

- 

- 

- 

AH-CV 

- 

- 

- 

- 

- 

SH-AB 

- 

0.000 

0.000 

- 

SH-CV 

- 

- 

0.060 

EO-AB 

- 

0.060 

EO-CV 

- 

COMBINED-TASK  RATINGS 

AH-AB 

AH-CV 

SH-AB 

SH-CV 

EO-AB 

EO-CV 

AH-AB 

- 

0.136 

- 

- 

- 

- 

AH-CV 

- 

- 

- 

- 

- 

SH-AB 

- 

0.035 

0.035 

- 

SH-CV 

- 

- 

0.035 

EO-AB 

- 

0.035 

EO-CV 

- 
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